Anthropics API Fundamentals

Getting Started

pip install anthropic 

from anthropic import Anthropic

client = Anthropic(
    api_key=my_api_key
)

our_first_message = client.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=1000,
    messages=[
        {"role": "user", "content": "Hi there! Please write me a haiku about a pet chicken"}
    ]
)

print(our_first_message.content[0].text)

Messages Format

• role: A string indicating the role of the message sender. It can be either "user" (for messages sent by the user) or "assistant" (for messages sent by Claude).

• content: A string or list of content dictionaries representing the actual content of the message. If a string is provided, it will be treated as a single text content block. If a list of content dictionaries is provided, each dictionary should have a "type" (e.g., "text" or "image") and the corresponding content. For now, we'll leave content as a single string.

messages = [
    {"role": "user", "content": "Hello Claude! How are you today?"}
]

messages = [
    {"role": "user", "content": "Hello Claude! How are you today?"},
    {"role": "assistant", "content": "Hello! I'm doing well, thank you. How can I assist you today?"},
    {"role": "user", "content": "Can you tell me a fun fact about ferrets?"},
    {"role": "assistant", "content": "Sure! Did you know that excited ferrets make a clucking vocalization known as 'dooking'?"},
]

• id - a unique object identifier

• type - The object type, which will always be "message"

• role - The conversational role of the generated message. This will always be "assistant".

• model - The model that handled the request and generated the response

• stop_reason- The reason why the model stopped generating its response.

• stop_sequence- The sequence that caused the model to stop generating, if any.

• usage - information on billing and rate-limit usage. Contains information on:
• input_tokens - The number of input tokens that were used.
• output_tokens - The number of output tokens that were used.

Few-shot prompting

chat_log = [{"role": "user", "content": "Perform the role of a chatbot and respond to the user"},
            {"role": "assistant", "content": "Hello! How can I help you today?"}]

chat_input = ""

print("Welcome to the chatbot! Type 'exit' to end the conversation.")

while chat_input.lower() != "exit":
    chat_input = input("You: ")
    chat_log.append({"role": "user", "content": chat_input})
    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=500,
        messages=chat_log
    )
    print("Bot: " + response.content[0].text)
    chat_log.append({"role": "assistant", "content": response.content[0].text})

print("Goodbye! Have a great day!")

Models

• Coding: Claude 3.5 Sonnet writes, edits, and runs code autonomously, streamlining code translations for faster, more accurate updates and migrations.

• Customer support: Claude 3.5 Sonnet understands user context and orchestrates multi-step workflows, enabling 24/7 support, faster responses, and improved customer satisfaction.

• Data science & analysis: Claude 3.5 Sonnet navigates unstructured data, generates insights, and produces visualizations and predictions to enhance data science expertise.

• Visual processing: Claude 3.5 Sonnet excels at interpreting charts, graphs, and images, accurately transcribing text to derive insights beyond just the text alone.

• Writing: Claude 3.5 Sonnet represents a significant improvement in understanding nuance and humor, producing high-quality, authentic, and relatable content.

Picking a model

• Capabilities
• The first and foremost consideration is whether the model possesses the necessary capabilities to handle the tasks and use cases specific to your application. Different models have varying levels of performance across different domains, such as general language understanding, task-specific knowledge, reasoning abilities, and generation quality. It's essential to align the model's strengths with the demands of your application to ensure optimal results.

• Speed
• The speed at which a model can process and generate responses is another critical factor, particularly for applications that require real-time or near-real-time interactions. Faster models can provide a more responsive and seamless user experience, reducing latency and improving overall usability. However, it's important to strike a balance between speed and model capabilities, as the fastest model may not always be the most suitable for your specific needs.

• Cost
• The cost associated with using a particular model is a practical consideration that can impact the viability and scalability of your application. Models with higher capabilities often come with a higher price tag, both in terms of API usage costs and computational resources required. It's crucial to assess the cost implications of different models and determine the most cost-effective option that still meets your application's requirements.

One approach: start with Haiku

Evaluating and upgrading

Model Parameters

max_tokens

• API limits & Cost: The number of tokens in your input text and the generated response count towards the API usage limits. Each API request has a maximum limit on the number of tokens it can process. Being aware of tokens helps you stay within the API limits and manage your usage efficiently.

• Performance: The number of tokens Claude generates directly impacts the processing time and memory usage of the API. Longer input texts and higher max_tokens values require more computational resources. Understanding tokens helps you optimize your API requests for better performance.

• Response quality: Setting an appropriate max_tokens value ensures that the generated response is of sufficient length and contains the necessary information. If the max_tokens value is too low, the response may be truncated or incomplete. Experimenting with different max_tokens values can help you find the optimal balance for your specific use case.

Stop sequences

stop_sequence allows us to provide the model with a set of strings that, when encountered in the generated response, cause the generation to stop. They are essentially a way of telling Claude, "if you generate this sequence, stop generating anything else!”. For example:

response = client.messages.create(
    model="claude-3-haiku-20240307",
    max_tokens=500,
    messages=[{"role": "user", "content": "Generate a JSON object representing a person with a name, email, and phone number ."}],
    stop_sequences=["}"]
)
print(response.content[0].text)

The response does not include the stop_sequence parameter

 

Temperature

The `temperature` parameter is used to control the "randomness" and "creativity" of the generated responses. It ranges from 0 to 1, with higher values resulting in more diverse and unpredictable responses with variations in phrasing. Lower temperatures can result in more deterministic outputs that stick to the most probable phrasing and answers.Temperature has a default value of 1.

 

When generating text, Claude predicts the probability distribution of the next token (word or subword). The temperature parameter is used to manipulate this probability distribution before sampling the next token. If the temperature is low (close to 0.0), the probability distribution becomes more peaked, with high probabilities assigned to the most likely tokens. This makes the model more deterministic and focused on the most probable or "safe" choices. If the temperature is high (closer to 1.0), the probability distribution becomes more flattened, with the probabilities of less likely tokens increasing. This makes the model more random and exploratory, allowing for more diverse and creative outputs.

System Prompt

• It's optional but can be useful for setting the tone and context of the conversation.

• It's applied at the conversation level, affecting all of Claude's responses in that exchange.

• It can help steer Claude's behavior without needing to include instructions in every user message.

def generate_questions(topic, num_questions=3):
    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=500,
        system=f"You are an expert on {topic}. Generate thought-provoking questions about this topic.",
        messages=[
            {"role": "user", "content": f"Generate {num_questions} questions about {topic} as a numbered list."}
        ],
        stop_sequences=[f"{num_questions+1}."]
    )
    print(response.content[0].text)

Streaming

How to use streaming?

stream = client.messages.create(
    messages=[
        {
            "role": "user",
            "content": "Write me a 3 word sentence, without a preamble.  Just give me 3 words",
        }
    ],
    model="claude-3-haiku-20240307",
    max_tokens=100,
    temperature=0,
    stream=True,
)

• MessageStartEvent - A message with empty content

• Series of content blocks - Each of which contains:
• A ContentBlockStartEvent
• One or more ContentBlockDeltaEvents
• A ContentBlockStopEvent

• One or more MessageDeltaEvents which indicate top-level changes to the final message

• A final MessageStopEvent

• end="": By default, the print() function adds a newline character (\n) at the end of the printed text. However, by setting end="", we specify that the printed text should not be followed by a newline character. This means that the next print() statement will continue printing on the same line.

• flush=True: The flush parameter is set to True to force the output to be immediately written to the console or standard output, without waiting for a newline character or the buffer to be filled. This ensures that the text is displayed in real-time as it is received from the streaming response.

Other streaming events

• Ping events - streams may also include any number of ping events.

• Error events - you may occasionally see error events in the event stream. For example, during periods of high usage, you may receive an overloaded_error, which would normally correspond to an HTTP 529 in a non-streaming context.

event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

TTFT - Time to first token