Architecting ML Infrastructure for AI-Powered Education

HMH was building an AI-powered personalization product for K-8 education, the product that ships today as HMH Pulse^TM. Its machine learning models needed a real path to production. The data science work lived in a legacy account where everything was hand-built. There was no governed route from training to registry to serving, and no isolation between experimentation and production. Onboarding new ML work took months to a year.

The launch date made it harder. Back to School is the one deadline in education that does not move. The models had to be serving in production before the school year started.

We built a multi-tenant ML Ops platform on AWS SageMaker, designed so the first tenant would prove the pattern for every tenant after it. I spent months embedded with the Learning Sciences Engineering teams, learning how their scientists actually work while writing the Terraform implementation. That engagement almost certainly saved us two years of rework. In the process of this project, I led the design and implementation of:

• Dedicated training and inference accounts (training-prod, inference-nonprod, inference-prod), isolated by design, all of it Infrastructure-as-Code
• Declarative tenancy: a new tenant is an entry in a list plus one Terraform file. Modules generate the SageMaker domains, model registries, CI runner roles, secrets wiring, and least-privilege IAM, 10+ roles per tenant
• Tenant isolation enforced with cross-account IAM, KMS encryption, and VPC endpoints, so one tenant's models and data never cross into another's
• A standardized CI pipeline covering train, register, promote, and serve. Models promote from development through integration to production, and there is no path that skips a stage
• Self-service by default, with 50+ pages of user docs and operator runbooks so teams could onboard without a ticket queue

I was the technical lead and program driver on the platform side. No project manager was ever assigned, so I picked the coordination up myself. I ran the cross-team Scrum of Scrums with the Learning Science Engineers and Data Scientists, and recorded and published every session. I authored the acceptance criteria for all six phases and ran every demo and handoff. Architecture review was mine to present as well.

When the SVP needed a single board slide nine months in, I framed the platform as three capabilities: tenant isolation, cross-account model promotion, and self-service provisioning. The investment held, and I have watched the SVP reuse that framing since.

When SageMaker fought us on SDK inconsistencies, endpoint configuration, and service limits, I worked with AWS specialists directly until we had answers. We implemented around the inconsistencies and opened feature requests with AWS for the rest.

The first three models shipped on time and powered the personalization product through its beta school year. Other models have since been developed on the same foundation, and the platform became the governed pattern for ML at the organization.

No one owned the coordination across teams, so I picked it up. Running the Scrum of Scrums, publishing every session, and writing acceptance criteria mattered as much to shipping as any Terraform module.

Hard lines protect architecture under pressure. With the deadline bearing down, the tempting shortcut was to reach back into the legacy account from the new environments. Saying no to that, clearly and early, kept the isolation model intact and made the platform trustworthy after launch.

Self-service has a boundary. Enabling a team means giving them working defaults, docs, and templates. Holding that line kindly when they ask you to take over their backlog is part of the job.