Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization

Generate synthetic doctor-patient conversations to overcome the scarcity of long-form audio data. This action pack guides you in building a pipeline for training and evaluating AI models for long-context audio summarization.

intermediate2 hours5 steps

The play

Define Conversation Parameters
Outline the specific scenario for your synthetic conversation. Include details like patient demographics, chief complaint, medical history, doctor's specialty, and the desired length of the interaction. This provides context for script generation.
Generate Dialogue Scripts
Use a Large Language Model (LLM) to generate realistic doctor-patient conversation scripts based on the parameters defined in Step 1. Ensure the script includes natural turns, medical terminology, and appropriate emotional tones.
Synthesize Audio from Scripts
Convert the generated dialogue scripts into audio files using a Text-to-Speech (TTS) model. Assign distinct synthetic voices for the doctor and patient to simulate a real conversation. Consider using open-source TTS libraries or cloud-based services.
Introduce Realism and Variation
Enhance the synthetic audio by adding realistic elements. This includes introducing slight pauses, varying speech rates, adding background ambient noise (e.g., hospital sounds, room tone), and subtle intonation changes to mimic natural human speech.
Evaluate and Iterate
Assess the quality, realism, and medical accuracy of the generated synthetic conversations and audio. Use both automated metrics (e.g., speech recognition accuracy on synthetic audio) and human review. Refine your parameters, LLM prompts, and TTS settings based on feedback to improve data quality.

Starter code

import openai

openai.api_key = "YOUR_OPENAI_API_KEY" # Replace with your actual API key

def generate_simple_script():
    prompt = """Generate a short, realistic doctor-patient conversation script. The patient is a 60-year-old male with knee pain. The doctor is an orthopedic specialist. The conversation should be about the patient's symptoms and the doctor's initial assessment.

Patient: "Good morning, Doctor. My knee has been bothering me for weeks."
Doctor: "Please, have a seat. Tell me more about your pain."
"""
    response = openai.chat.completions.create(
        model="gpt-4o", # Or another suitable LLM
        messages=[
            {"role": "system", "content": "You are a helpful assistant that generates medical dialogues."},
            {"role": "user", "content": prompt}
        ],
        max_tokens=300,
        temperature=0.7
    )
    return response.choices[0].message.content

print(generate_simple_script())

Source

Paperarxiv.org