Trace and Debug Your First LLM App with LangSmith

1. 1
   Get Your LangSmith API Key

   Sign up on the LangSmith website. Once logged in, navigate to the 'Settings' page (bottom-left icon) and then to the 'API Keys' tab. Create a new API key and copy it. This key allows your application to send data to LangSmith.
2. 2
   Configure Environment Variables

   Set environment variables in your terminal or development environment. LangSmith uses these to automatically instrument your LangChain application. You'll also need an OpenAI API key for the example code.
3. 3
   Run a Traced Application

   Run the Python script from the 'starter' section below. Because the LANGCHAIN_TRACING_V2 environment variable is set to true, LangChain will automatically capture the entire execution and send it to your LangSmith project.
4. 4
   Inspect the Trace in LangSmith

   Navigate to your project in the LangSmith UI. You will see a new run corresponding to the script you just executed. Click on it to view a detailed, hierarchical trace of the chain, including inputs, outputs, latency, and token usage for each step.
5. 5
   Add a Run to a Dataset

   From the trace view of your run, click the 'Add to Dataset' button. You can create a new dataset or add to an existing one. This saves the run's inputs and reference outputs, forming the basis for regression testing and evaluation.

# 1. Save this code as your_langsmith_script.py
# 2. Install required libraries:
# pip install langchain langchain-openai langsmith

# 3. Make sure your environment variables are set (see Step 2)

import os
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Check if API keys are set (optional but good practice)
if os.getenv("LANGCHAIN_API_KEY") is None or os.getenv("OPENAI_API_KEY") is None:
    print("Please set LANGCHAIN_API_KEY and OPENAI_API_KEY environment variables.")
    exit()

# 1. Define a simple chain
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

model = ChatOpenAI(model="gpt-3.5-turbo")

output_parser = StrOutputParser()

chain = prompt | model | output_parser

# 2. Invoke the chain
# This run will be automatically traced to your LangSmith project
print("Invoking the chain...")
result = chain.invoke({"input": "What is the capital of France?"})

print("\nResult:")
print(result)
print("\nTrace captured! Check your project in LangSmith.")