Amazon Kendra redux — Large Language Model edition to crack the CBSE 12th physics exam ;-)
Time flies .. It was in 2020, and I had just got my hands onto Amazon Kendra, it was peak covid, and our eldest daughter was in 10th grade of school and this was a deadly combination to try using Kendra for this dad-daughter use-case 😃 So, I wrote this personal blog which leveraged Kendra to answer questions for Physics !! I updated my blog last year to indicate the evolution of Kendra with the addition of new features and availability in the AWS Mumbai region ..
Well, now we are in 2023 and the age of Generative AI (GenAI) and large language models (LLMs), and can you leverage Amazon Kendra in the scenario ? This question was answered in an awesome AWS blog, that was published recently which uses a technique called Retrieval Augmented Generation (RAG) and combines the capabilities of Amazon Kendra with LLMs from Amazon Sagemaker Studio Jumpstart to generate a conversational experiences over your content (the emphasis is on your content — personal/enterprise content and not content found on the Internet) !!
So, in this very short blog, I have tried the solution from the AWS blog and revisit my favorite “dad-daughter” use-case again with her CBSE Class 12 physics text books 😄 The difference, since it is May 2023 and her 12th grade board exams are already over, I will try with some actual exam questions and use the Amazon Kendra S3 connector to index her physics textbook files with the help of the LLMs from Amazon Sagemaker Studio Jumpstart !! I am also dabbling in Resilience related conversations these days and hence will try another test with a Kendra Web Crawler connector crawling some AWS Resilience related service documentation ..
As always, these personal blogs are meant to satisfy my technical curiosity and my itch to do some hands-on stuff to learn new concepts 😃 The AWS documentation and blogs are the final authority, and you will incur charges when you try these things in your AWS account. Again, some of the concepts are proof of concept, and may need more deep dive for production ..
Lets get started ..
The AWS blog has all the details of the solution, but in summary — the blog shows how to implement a Retrieval Augmented Generation (RAG) workflow by combining the capabilities of Amazon Kendra with LLMs from Amazon Sagemaker Studio Jumpstart to create GenAI applications providing conversational experiences over my content. Amazon SageMaker JumpStart offers pre-trained, open-source models for various problem types, giving you a head start in your ML journey.
The architecture of the demo from the AWS blog that I will be setting up to test my use-cases. I tested with two LLM’s — FLAN T5 XL and FLAN T5 XXL from Hugging Face.
first, get your data organized
Amazon Kendra now supports many connectors to add data sources to your Amazon Kendra index. I used the following connectors for my usecases:
- Amazon Kendra S3 connector — to the Amazon S3 bucket which had her Class 12 physics pdf files downloaded from the NCERT website — https://ncert.nic.in/textbook.php?leph1=0-8
- Amazon Kendra Web Crawler, pointing to a small subset of the following AWS websites to index:
second, set the required LLM’s in Amazon Sagemaker Studio Jumpstart
I setup two model endpoints in Amazon Sagemaker Studio Jumpstart— FLAN-T5 XXL and FLAN-T5 XL — Text2Text Generation models built upon a T5 model from Hugging Face. In the future, I may test it with Anthropic , OpenAI and of course, Amazon Bedrock (when I get access).
third, set up the application code and let the testing begin 😅
Setting up the code from the https://github.com/aws-samples/amazon-kendra-langchain-extension is fairly easy, and I used AWS Cloud9 (as suggested in the blog) for my development environment. I love open-source frameworks like LangChain , which provide modules to integrate with the LLM of choice.
Tip: Ensure that you have set the correct environment variables in Cloud9 terminal and the security group of the Cloud9 EC2 instance is open for port 8501 to access the streamlit web app.
Fingers crossed 🤞 I used a few questions from the 2023 CBSE class 12 physics question paper from the web ..
and one more ..
and got some great results for the CBSE physics questions !!
I want to try the other use-case, where Amazon Kendra used the web crawler for the Resilience related documentation ..
Tip: Make sure you point the Resilience Index to the right data source, when you run the streamlit app !!
As you can see, with this solution as outlined in the AWS blog, you can leverage the power of Amazon Kendra, which has out-of-the-box semantic search capabilities for state-of-the-art ranking of documents and passages, and its pre-built connectors to popular data sources such as Amazon Simple Storage Service (Amazon S3), SharePoint, Confluence, and websites, and supports common document formats such as HTML, Word, PowerPoint, PDF, Excel, and pure text files with the power of LLMs hosted in Amazon SageMaker Studio Jumpstart to create state-of-the-art GenAI applications providing conversational experiences.
Conclusion
While I used some simple usecases here, the concepts and solution can be used for actual real-world use-cases like in Contact center, Financial services, IT help desk, self service applications and many more.
That’s it folks, hope this blog was useful and informative ..
Resources
- The AWS blog is the genesis for my personal blog, I am really indebted to the authors for creating such a simple solution, which is easy to follow and use — https://aws.amazon.com/blogs/machine-learning/quickly-build-high-accuracy-generative-ai-applications-on-enterprise-data-using-amazon-kendra-langchain-and-large-language-models/
- Another AWS blog demonstrating Question answering using Retrieval Augmented Generation with foundation models in Amazon SageMaker JumpStart, without using Amazon Kendra — https://aws.amazon.com/blogs/machine-learning/question-answering-using-retrieval-augmented-generation-with-foundation-models-in-amazon-sagemaker-jumpstart/
- Amazon Kendra workshop, if you are new to Kendra — https://catalog.us-east-1.prod.workshops.aws/workshops/df64824d-abbe-4b0d-8b31-8752bceabade/
- Amazon Sagemaker Studio Jumpstart — https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html
