Community powered AI “Superclusters”
The use of AI in enterprise products and services is booming (Forbes on how businesses are using AI in 2023 article). However, in order to build and deploy modern AI models, abundant data, quality data and additional compute resources are needed.
In this all-new implementation, FedML enables Theta Edge Network community members to contribute their personal preferences and compute resources to train and deploy AI models for personalized content recommendation for Theta.tv viewers.
The screenshot below shows the community of edge nodes around the world and the edge compute dashboard of the updated Theta Edge Node, which can be downloaded from the following links:
- Windows and macos installer: https://www.thetatoken.org/join
- Docker image: https://docs.thetatoken.org/docs/theta-edge-node-run-with-docker
Decentralized machine learning has significant venture activity in 2023. And given the trend of AI Goes Decentralized, “super clusters” based on community provided compute resources will empower the next generation of models and user experiences. Theta edge nodes powers such a decentralized compute approach and rewards Theta’s highly engaged community. The reward comes in two forms:. there will be better user experience on Theta.tv. and the node community will be rewarded for their contributions (i.e., enabling AI by the community, for the community).
FedML has always been about open and collaborative artificial intelligence, with collaboration at the heart of FedML’s infrastructure and its easy to use MLOps user interface. This new implementation with Theta takes that collaboration to a new level and scale.
To deliver this Web3 x AI platform to the community, the two companies built an interdisciplinary team of experts in machine learning (ML), blockchain, computer vision, natural language processing, security (zero-knowledge proof), and privacy.
The FedML clients are integrated into the Theta Edge nodes and will periodically pull data from the Theta.tv backend. The FedML training platform will then utilize those Theta Edge Nodes and their data to train, fine-tune, and serve the video recommendation model as shown in the diagram below:
The FedML module can be used in Theta Edge Node Windows and MacOS GUI versions, as well as the Linux Docker container. Each recommendation model is trained on 100-1,000 samples via Edge Nodes before it is used to make user recommendations.
Decentralized Training powered by FedML MLOps
FedML provides the Theta Labs team and other community computing pioneers a simple to use MLOps platform & UI for training, observing, and serving (blog with introduction example). Some of FedML’s screens used by the Theta recommendation engine are shown below. For training, the edge device list will be a list of all those in the community who are providing their unused compute. Each time a training run is completed, FedML provides details such as a status, training compute time for each edge, observability, and logs.
Federated Model Serving
FedML’s serving platform will deploy the model to the new community-driven geo-distributed compute.
FedML federated model serving platform provides model as a service for diverse AI verticals, powered by federated model inference via geo-distributed cloud computing resources. In the model serving platform, FedML mainly provides four modules: model card, computing resource, inference endpoint, and monitoring.
Each training run automatically produces a model and a model card which can be converted into an inference endpoint. The following screen shows a deployed endpoint and an example curl command.
What’s next?
The applications of this technology are enormous for any video platform, beginning with the Theta community. Any media company can use the power of decentralized ML to better serve their users the videos they want, increasing watch times and user engagement. The applications extend well beyond video recommendation. Examples include listing news stories that are more relevant to you, or a streaming platform recommending the movies which may be most helpful and relevant to you - saving you from a painful hour of scrolling through & viewing hundreds of options. New Generative AI may also support the video experience with automated summaries or generative video content to help suggest gaming strategy. FedML already supports training & serving such generative AI models. The technology can also be used to improve how video ads are served, a market that is projected to be $177 billion in 2023 according to Statista.
FedML and Theta Labs are both based in Silicon Valley, are building the next generation of MLOps with distributed AI training & serving MLOps..
Theta Labs pioneered Theta Network, the next-generation Layer1 purpose-built blockchain for video and entertainment, which powers THETA.tv, a decentralized live streaming platform, and ThetaDrop, an NFT marketplace partnered with Katy Perry, ABS-CBN, Jukin Media, Fuse Media and others. Theta’s enterprise validator and governance council is led by global leaders including Google, Samsung, Sony, Creative Artists Agency (CAA), Binance, Blockchain Ventures, DHVC, and gumi. Theta Labs Github is here and Medium blog is here.
FedML’s MLOps platform (www.fedml.ai) empowers any model to be deployed using distributed training & serving on-prem, cloud, multi-cloud, or geo-distributed compute. The models FedML deploys range from simple neural networks, CNNs, on up through modern Large Language Models. In all cases, FedML’s is working with its clients to implement FedML’s low-code MLOps and LLMOps that scale artificial intelligence deployments. To learn more, read FedML’s getting started blog post here, and FedML’s open source resources on GitHub here.
