MoRight: Motion Control Done Right

MoRight enables creation of AI-generated videos with precise, disentangled control over object motion and camera viewpoints. This ensures physically plausible scene dynamics, empowering users to produce highly customized content for applications like film, gaming, and synthetic data generation.

intermediate30 min5 steps

The play

Understand Disentangled Control
Grasp the core principle of separating object motion specifications from camera perspective adjustments. Recognize that MoRight allows independent control over 'what moves' and 'how it's viewed'.
Define Object Actions
Specify the desired movements, behaviors, and interactions for individual objects or characters within your video scene. Detail their paths, speeds, and any specific actions (e.g., 'walk', 'jump', 'interact').
Choose Camera Viewpoints
Independently select the camera angles, movements (e.g., pan, zoom, dolly), and framing. Dictate how the scene unfolds from a visual perspective without directly affecting object physics.
Generate Controlled Video
Utilize a system based on MoRight's principles to synthesize the video. The system will integrate your disentangled motion and camera controls to produce a physically plausible and customized output.
Review and Iterate
Evaluate the generated video for accuracy, realism, and adherence to your artistic vision. Refine your object motion and camera perspective inputs as needed to achieve the desired outcome.

Starter code

{
  "video_generation_config": {
    "scene_description": "A character walking through a forest.",
    "object_motion": {
      "character_1": {
        "action": "walk",
        "path": [
          {"time": 0, "position": [0,0,0]},
          {"time": 5, "position": [10,0,0]},
          {"time": 10, "position": [20,0,0]}
        ],
        "speed": "moderate"
      }
    },
    "camera_perspective": {
      "type": "dolly",
      "start_position": [5, 2, 10],
      "end_position": [15, 2, 5],
      "target_object": "character_1",
      "field_of_view": "wide"
    },
    "output_settings": {
      "resolution": "1920x1080",
      "frame_rate": 30
    }
  }
}

Source

Paperarxiv.org