Comprehensive blogpost creation Application using Open Source Meta Llama2–13b, Serverless architecture, Amazon Bedrock, AWS Lambda, API Gateway, S3 bucket and Amazon CloudWatch.

Project overview:

In this hands-on, I create a proof of concept (PoC) ‘blogpost generation application’ that uses Llama2–13b-chat-v1 to generate blog post. This project aims to use Amazon Bedrock as a managed service for rapid prototyping deployment of AI applications. I used postman to call application.

Why is this project important?

This is a basic while fundamental in understanding of deploying generative AI. An LLM application is comprised of multiple components. Knowing these components, their architecture and more importantly, providing a functioning solution, requires understanding of each component. A good practice is to create a prototype and later scale it up. This project focuses on a serverless architecture that provides simple, cost effective and scalable system for quick deployments.

The critical components used in this project are:

- Meta Llama2–13b-chat-v1for blog generation.

- Amazon Bedrock for integrating application with LLMs with single API endpoint.

- Amazon Lambda for serverless architecture.

- Amazon Gateway for API endpoints.

- Amazon S3 for storing generated blogpost.

- AWS CloudWatch for log generation.

Architecture:

In this serverless event-driven architecture, a Post HTTP request is invoked in postman. This request is an event that contains the body which is generating the request. The requests are routed through Amazon API Gateway to the Lambda function. Lambda function processes the request by sending a prompt to the bedrock. Bedrock routes the request to the specified LLM using single API. The LLM generates the response and sends it back to the Bedrock and back to the Lambda function. Lambda runs the second function that is storing the content in s3 bucket.

Prerequisite:

This hands-on requires experience working with Amazon Web Services (AWS) services, Python, Large Language Models (LLMs), web development and VS code (optionally).

Requirements:

- AWS account.

- Python version 3.12

- Conda for virtual environment creation. Optionally, you can use python to create virtual environment.

project implementation steps:

1- Open your VS code (or cloud shell in AWS) and create folder “bedrock-lambda”.

2- Start the terminal and create Virtual Environment and activate it.

Conda create -p  venv/
Conda activate

3- Create app.py and requirements.txt.

In app.py create your project.

# import the libraries
import boto3
import botocore.config
import json

from datetime import datetime

# defining the main function of creating blogpost.
def blog_generate_using_bedrock(blogtopic=str)->str:
    prompt=f"""<s>[INST]Human: write a 200 blogpost on the topic {blogtopic}
    Assistant:[/INST]
"""
    # Body parameters can be set to desired output. 
    body={
        "prompt":prompt,
        "max_gen_len":512,
        "temperature":0.5,
        "top_p":0.9

    }

    # Try Exception blobk helps catching errors.
    try:
        bedrock=boto3.client("bedrock_runtime", region="us-east-1",
                             config=botocore.config.Config(readtimeout=300, retries={'max_attempts':3}))

        response = bedrock.invoke.model(body=json.dumps(body), modelId="meta.llama2-13b-chat-v1")

        response_content=response.get('body').read()
        response_data = json.loads(response_content)
        print(response_data)

        blog_details = response.get['generation']
        return blog_details
    except Exception as e:
        print(f"Error generating blog:{e}")
        return ""

    # Define a function that stores the generated blogpost in s3 bucket.
    def save_blog_details_s3(s3_key,s3_bucket,generate_blog):
        s3 = boto3.client('s3')

        try:
            s3.put_object(Bucket = s3_bucket, Key= s3_key, Body= generate_blog)
            print("Code saved in s3")
        
        except Exception as e:
            print("Error when saving the code to s3")

    # Define lambda handler function. 
    def lambda_handler(event,context):
        event = json.loads(event['body'])
        blogtopic=event['blog_topic']

        generate_blog= blog_generate_using_bedrock(blogtopic=blogtopic)

        if generate_blog:
            current_time=datetime.now().strftime('%H%M%S')
            s3_key=f"blog-output/{current_time}.txt"
            s3_bucket='aws_bedrock_course100'
            save_blog_details_s3(s3_key,s3_bucket,generate_blog)

        else:
            print("No blog was generated")

        return{
        'statusCode':200,
        'body':json.dumps('Blog Generation is completed')
    }

4- In requirements.txt, add below libraries.

Boto3 

5- Configure Amazon Bedrock service.

In AWS console, search Amazon Bedrock, click on ‘get started’. On the left panel, click on Model access, and select the models you would like to use. Request for access and AWS will grant access shortly. For this project, we will use open-source Llama 2 Chat 13B.

Take a note of the API request snippet of your model. You need the model ID in your code.

6- Create Lambda function.

In console search lambda and create function.

Go to lambda > functions > awsappbedrock

In the code section paste your app.py code.

In the configuration tab, set the timeout to 3 minutes.

7- Lambda library configuration:

Lambda function does not update boto library. We need upload boto3 as a layer to lambda function. To do that:

- Create a folder boto3, then create a subfolder python in VScode, and install boto3 using

pip install boto3 -t python/

zip the folder and upload it as a layer to lambda.

In lambda page, go to the last option, add layer, name the layer , use upload file option and add the layer. Lambda will use the libraries in the layer.

For runtime, you can use python 3.12, 11 and 10.

Finally deploy your lambda function.

8- Create API gateway:

In AWS console, create an HTTP API.

• Then you need to create Route for the API. Route will be of type POST and the name is /blog-generation.

• Attach integration: attach to your lambda function.

Select the environment you would like to create the API gateway for. In our case, I use it for development environment, so I named it “dev” stage.

And deploy the API gateway.

Invoke URL indicates that the API gateway invoke url is integrated with lambda function.

9- create S3 bucket.

Create a unique name for your bucket and ensure you add it to the body of your code.

10- Test the function with postman.

We get an error message status:404 Not Found. This can have different reasons, such as API gateway misconfiguration, wrong deployment stage and lack of sufficient permissions for lambda to send request to Amazon Bedrock.

11. Add admin role to the lambda function.

In production environment, make sure you follow your companies permission policies.

Run the test with postman again.

12. Check AWS CloudWatch logs.

You will see blog generated as expected. Also, below the blog, you see “the code saved to s3”. Let’s check it in s3.

It is important to enable AWS CloudWatch for logs. The logs can provide details of processes and errors.

13. Checking blog saved in s3.

Download the object and read it in a text editor. Every time you invoke a request or simply prompt the LLM, a new content will be created in s3 bucket with new object name.

Read the output in text editor.

Conclusion and key insights.

This hands-on provides a clear understanding of how an application integrates with an LLM. Amazon Bedrock is a managed service provides easy integration with other services while takes care of infrastructure, scalability, compliance, and security, and let us focus more on application customization and fine tuning. The filed of AI is constantly changing and managed services such as Amazon Bedrock will play critical roles in complex architecture.