SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis

Leverage SceneCritic's symbolic evaluation to objectively assess 3D indoor scene synthesis. This method analyzes scene graphs for geometric and semantic consistency, providing robust feedback beyond subjective LLM/VLM judges.

intermediate30 min4 steps

The play

Recognize LLM/VLM Evaluation Flaws
Understand that traditional 3D scene evaluation using LLM/VLM judges is often subjective and sensitive to rendering parameters or prompt phrasing, leading to unreliable assessments.
Generate Symbolic Scene Representations
Instead of pixel-based outputs, represent your 3D scenes as symbolic structures like scene graphs. Include details on object types, positions, orientations, and their relationships (e.g., 'table supports cup').
Apply Symbolic Consistency Checks
Implement or integrate an evaluator (following SceneCritic's principles) that assesses these symbolic representations. Focus on geometric consistency (e.g., valid object placement, support relationships) and semantic consistency (e.g., appropriate object types, functional context).
Utilize Objective Feedback
Use the clear, interpretable feedback from symbolic evaluation to identify and correct specific errors in your 3D scene generation models, accelerating development and improving realism.

Starter code

{
  "scene_id": "office_scene_01",
  "objects": [
    {
      "id": "obj_desk_01",
      "type": "desk",
      "position": [0, 0, 0],
      "size": [1.5, 0.7, 0.8]
    },
    {
      "id": "obj_laptop_01",
      "type": "laptop",
      "position": [0, 0.6, 0.81],
      "size": [0.3, 0.2, 0.03]
    }
  ],
  "relationships": [
    {
      "predicate": "on",
      "subject": "obj_laptop_01",
      "object": "obj_desk_01"
    }
  ]
}

Source

Paperarxiv.org