2023:Program/Technology/3UYGTF-Lift Wing: WMF's machine learning model serving infrastructure

The following is a mirror of what are publicly available details of submission made at the call for proposal platform, Pretalx. Any changes here may be overwritten by a bot while syncing updates from Pretalx (3UYGTF). Changes are monitored and may be updated into Pretalx if they do not change the context of the submission (i.e. correcting typographical errors).

Title: Lift Wing: WMF's machine learning model serving infrastructure

Speakers:

Machine Learning Team

The machine learning team is responsible for designing, building and maintaining the foundation’s machine learning infrastructure. The team also trains, deploys and manages machine learning models created or requested by Wikimedia teams or Wiki communities while at the same time trying to develop best practices for applied ethical machine learning.

Pretalx link

Etherpad link

Room: Room 310

Start time: Sat, 19 Aug 2023 10:30:00 +0800

End time: Sat, 19 Aug 2023 11:00:00 +0800

Type: No (pretalx) session type id specified

Track: Technology

Submission state: confirmed

Duration: 30 minutes

Do not record: false

Presentation language: en

Abstract & description

Abstract

Lift Wing hosts production machine learning models that empower wiki projects. It is a production Kubernetes cluster hosting KServe, a serverless inference service. Its purpose is to enable shipping machine learning models as scalable microservices by the community and the Wikimedia foundation to be used internally but also if needed made publicly available through the API Gateway.

Description

Lift Wing is a machine learning inference platform that is being developed as part of the modernization efforts for the Wikimedia movement’s machine learning infrastructure. The framework is designed to provide a unified platform for developing and deploying machine learning models supporting various Wikimedia projects.

The platform is built on top of KServe (previously known as KFServing) which started as part of Kubeflow but later became a standalone project. KServe handles the part of the machine learning lifecycle that happens after training a machine learning model. It enables deploying a trained model as a microservice to kubernetes. Each service automatically takes advantage of the capabilities that Kubernetes offers, allowing to scale capacity up and down according to the requested traffic. Furthermore models receiving traffic sporadically may stay idle not utilizing any resources until a client request comes, taking advantage of serverless inference.

Staying loyal to our movement means that transparency and explainability are indispensable: they play an important role in engaging with other community members and convincing them to use a model. To further advocate this, models deployed on Lift Wing need to be accompanied by a model card. A model card is a wiki page that provides information about a model’s development process, performance, limitations and ethical or other considerations.

Further details

Qn. How does your session relate to the event themes: Diversity, Collaboration Future?

The topic at hand has to do with WMF changing the way it does machine learning, in an effort to allow the community to easily build and ship machine learning models that empower Wiki projects.

Qn. What is the experience level needed for the audience for your session?

Some experience will be needed

Qn. What is the most appropriate format for this session?

• Onsite in Singapore
• Remote online participation, livestreamed
• Remote from a satellite event
• Hybrid with some participants in Singapore and others dialing in remotely
• Pre-recorded and available on demand