Introducing MediaGenerationKit: Hybrid Media Generation for Production Apps

A Swift package for integrating open-weight image and video models, with the flexibility to run locally or through the cloud.

Draw Things app started as a local-only solution for running image and video generation tasks efficiently on Apple Silicon. Over the years, we have focused on the quality of generation, the efficiency of inference, and, more recently, on how to enable people to safely and securely offload their generation tasks to our Cloud when their devices are too weak.

There are many providers nowadays offering model inference as a service. Most of them provide TypeScript SDKs, and some of them provide Swift packages. None of them offer hybrid local and remote generation support, which is increasingly essential for customers trying to balance reach, quality, and cost.

Today, we are releasing a preview of MediaGenerationKit, a Swift package that allows you to integrate the latest open-weight media generation models into your app. You can choose to run them locally, host inference stack on your own Cloud or through our Cloud. For model selection, you can directly use a Hugging Face link to reference a model, or use one of our internal model identifiers. It supports the same comprehensive generation configurations that the Draw Things app supports.

For a simple example, it is really just a few lines of code:

import Foundation
import MediaGenerationKit

@main
struct ExampleApp {
  static func main() async throws {
    try await MediaGenerationEnvironment.default.ensure(
      "flux_2_klein_4b_q8p.ckpt"
    )

    var pipeline = try await MediaGenerationPipeline.fromPretrained(
      "flux_2_klein_4b_q8p.ckpt",
      backend: .local
    )

    pipeline.configuration.width = 1024
    pipeline.configuration.height = 1024
    pipeline.configuration.steps = 4

    let results = try await pipeline.generate(
      prompt: "a cat in studio lighting",
      negativePrompt: ""
    )

    try results[0].write(
      to: URL(fileURLWithPath: "/tmp/cat.png"),
      type: .png
    )
  }
}

Unlike draw-things-cli, MediaGenerationKit allows you to offload generation tasks either to your local gRPC server or to our Cloud Compute backend. For our Cloud Compute backend, you can request an API key with your existing Draw Things+ account, or purchase one, at api.drawthings.ai/dashboard. The default API key supports up to 20 generation tasks per month for the Free tier and up to 200 generation tasks per month for the Draw Things+ tier. Beyond that, you would use the pay-as-you-go API flow.

You can integrate MediaGenerationKit today using Swift Package Manager:

import PackageDescription

let package = Package(
  name: "MyApp",
  dependencies: [
    .package(
      url: "https://github.com/drawthingsai/media-generation-kit.git",
      revision: "37e0b70092ad1a0c1d4c6b24f16e17b73f1b1fb3"
    )
  ],
  targets: [
    .executableTarget(
      name: "MyApp",
      dependencies: [
        .product(name: "MediaGenerationKit", package: "media-generation-kit")
      ]
    )
  ]
)

To learn more about the project, read the README at https://github.com/drawthingsai/media-generation-kit