preface: i am not an ml engineer. i am just a lowly web dev dipping a toe in the deep water.
project: myself and a handful of colleagues had it in mind to make mlflow friendlier for JS devs who want to track and register models that run in browser or node.
so far, we have abstracted and generalized all 44 of the RESTful endpoints into modules organized thus:
├── model_registry
│ ├── model_registry.js
│ └── model_version_management.js
├── tracking_server
│ ├── experiment_management.js
│ ├── run_management.js
now we want to add more layers of abstraction.
a couple we’ve spun up so far:
retrainModelIfPerformanceDrifts(experimentName, baselineRunId, metricName, threshold, modelFunc, paramSpace)
and
withExperimentRun(experimentName, runName, callback)
we have several more bundled up, but we want to hear what the community has to say before we weigh in here.
question: for those of you who brush up against MLOps in your day-to-day and who either already work in JS or would if it made more sense to do it, what are some canned workflows and functionalities that would make your lives easier and your DX richer?
unsolicited suggestions, questions, and rude remarks are also welcome.