Can I use Anthropic Claude 3 Sonnet (on Amazon Bedrock) to help my kid get better grades (pass!!) in Hindi 🤣?
TL DR;
A quick experiment to check, if we can use an LLM (Large Language Model) like Anthropic Claude 3 Sonnet (on Amazon Bedrock) to help with Hindi lessons for our youngest kid. The answer was mostly an “yes”, to augment regular classroom teaching.
Well, here’s the backstory — the kids who are in CBSE (Central Board of Secondary Education) in India, have to study languages along with core subjects. Our youngest kid who is now in her sixth grade, has to study three languages — our choices were English as first language, a second language, which is Hindi in her case and a third language, which is Sanskrit. She is a generally very good in studies, arts (an amazing artist and plays the keyboard very well) as well as sports (tennis), but when it comes to languages, learning Hindi is becoming a major problem. I wont go into too much detail, but when you are a non-native speaker of any language, we sometimes struggle to learn new languages, especially if we don’t speak this language extensively at home or with friends (she can handle English and a little bit of Tamil and Kannada). While I can speak multiple Indian languages (a trait most Bangaloreans have 😊 !!), we are unable to help our kid with her class exercises and assignments in Hindi as the complexity of the Hindi lessons + grammar is increasing every year and is now beyond our capability !!
While it is too late this year, as her Hindi exam just got over, I have been thinking of various options — including tutors, as well as tech based solutions like leveraging the power of Generative AI, as a lot of Large Language Models now support multiple languages and not just English !! I have experimented with using AWS services like Amazon Kendra for helping our eldest kid with subjects like Physics a few years back, but with Generative AI now, it is a totally different world now.
What do we want to achieve?
Based on my interactions with our daughter, the solution should definitely augment her Hindi lessons at school, and help her in two main areas:
- Help with textbook exercises including answers to the questions at the end of each lesson in her textbook. She really needs help here ..
- We don't believe in rote learning. She also needs to understand the answers in English, and be able to listen to the generated answers in Hindi. This will help her to understand as well as remember the answers in Hindi. Learning is equally important to the first goal of passing exams, as just mugging the answers will not help her understand the language.
Movie Trivia: I am a movie buff, and there is an old classic Tamil movie called “Indru Poi naalai Vaa” (Don’t come today, come tomorrow), where there are some rip roaring comedy scenes in trying to learn Hindi. Recommended, if you know Tamil 🤣
what next? my experiments ..
I have been reading about advances and launches of various LLM’s built in/for India which support major Indian languages. OpenHathi from Sarvam AI and IndicBART from ai4bharat certainly look interesting and are available in the Hugging Face repository and can be deployed on Amazon SageMaker.
An admission, I am an Solutions Architect and a generalist with a background in applications/containers/serverless, and not a Data Scientist or an AI/Machine Learning specialist. Hence, while I can create simple ML notebooks and can program in multiple languages including python, I want to look at solutions. which do not involve deep ML stuff 😃
Given that both on these models are not yet available via Amazon SageMaker Jumpstart (a Machine learning (ML) hub with foundation models, built-in algorithms, and prebuilt ML solutions that you can deploy with just a few clicks), and they will probably be available soon and with my own limitations around diving deep into notebooks and ML, I decided to look at other high-level alternatives on a Saturday afternoon over the weekend 🤣
Anthropic Claude 3 (Claude is a family of state-of-the-art large language models developed by Anthropic) has been launched and has garnered a lot of very positive reviews. The Claude 3 family of models has three models — Opus, Sonnet and Haiku. You can read about these models at https://docs.anthropic.com/claude/docs/models-overview. and https://www.anthropic.com/news/claude-3-family. Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies including Anthropic. Sonnet and Opus are available in Bedrock, and Opus is expected soon ..
While reviewing the documentation of Claude 3, I observed that Claude 3 has “Claude 3 models offer improved fluency in non-English languages such as Spanish and Japanese, enabling use cases like translation services and global content creation”. Plus, all Claude 3 models can process and analyze visual input. These two data points, plus their availability Amazon Bedrock and how it is very easy to test the models on Amazon Bedrock via the AWS console, I decided to do quick proof of concept using Claude 3.
Of course, I am aware that Hindi is not explicitly mentioned in the Claude 3 documentation, I am still going ahead in doing a quick proof of concept.
My plan was to do a quick proof of concept, leveraging Claude 3 Sonnet and other AWS AI services — Amazon Translate to translate from Hindi to English and Amazon Polly to do a text to speech from Hindi.
She has two text books in Hindi, one on prose and poetry and the other textbook is on grammar. I selected a lesson called Mangal grah ki sair/A trip to Mars to do my proof of concept ..
My first prompt is to get a summary of the lesson:
I am a 12 year old child, and I am trying to understand this lesson. can you give me a summary of this lesson in Hindi? please stick to only the facts shared in the textbook.
The output seems good, at-least to my amateur eyes !! I am sure the prompt can be better, and we can tweak the prompt to give longer or shorter answers based on the marks for that question. I plan to use Amazon Translate to generate the English language translation of the summary and then use Amazon Polly to generate a text to speech using an Indian voice to deliver natural sounding native speaker.
Let us now try some of the questions asked in the lesson ..
Asking the questions individually and in Hindi gave better responses, with much lesser hallucination like these ones:
I think the above question is not entirely true, let me try with a different prompt 😃
Well, there were some instances the LLM could not get an answer .. Like some of the questions in section 1, “what is the occupation of ananya’s mother?” Is there an answer to this question from the textbook? Does she work or is an homemaker?
This needs more deeper analysis later, as to why this question cannot be answered.
While I take care of the “important” subjects like Hindi, English, History 🤣, the missus does a great job in covering Maths (🤕), Physics, Bio and Chem .. I could not resist doing a quick test on Claude Sonnet with a question from Geometry. It did a awesome job with proper explanations which made it easy for a sixth grader to understand the solutions. Now I know, whom to reach, if my wife is not available !!
I wanted to verify the answer, and I realized that while our kid got the formula and approach correct, she had done a silly mistake in copying the answer after division 😃 (I am guilty of this, and she probably has inherited my genes) !! Thank you Claude Sonnet.
backlog and next steps
This is still a work in progress, but I am excited at the possibilities. My findings based on this quick experiment over the weekend:
- Claude 3 Sonnet did an a great job of responding to several of the questions, much better than I could. This approach will certainly help us (the parents) as well my daughter to get a better grip on the subject. Hindi as a language is not explicitly mentioned in the documentation, but it seemed to work fairly well in my experiments.
- Using the right Prompt is very, very important, as the LLM can hallucinate and give out stuff not in the textbook or sometimes wrong answers. “trust but verify” should be the approach. Prompts in Hindi give out better answers, than asking questions in English. So, having a soft copy of the text book, where you can copy and paste the questions will be better approach.
- For Hindi, I feel confident of using this solution for grammar, letter writing and for creative writing for now.
- The combination of Amazon Polly and Amazon Translate is an awesome combination to complete the flow ..
Our kid has her annual summer break in a week, and hence I have plenty of time to iterate on my next steps ..
- When the mother of al LLM’s, Claude 3 Opus releases to the general public, evaluate all the tests again on Opus ..
- Build an functional app which stitches all these things together. I am thinking of building a simple Streamlit based app, which uses langchain. There are nice resources like this blog on medium to get started as well as code samples at the AWS GitHub samples repo.
- I would also like to test the above scenarios with other LLM’s like OpenHathi and IndicBart and whether they can also fit our requirement.
Finally, this was an effort to test something in a few hours in a sudden burst of serendipity on a weekend. Learning languages, in my view, are very important as much as core subjects. Again, to reiterate, classroom teaching and teachers are very, very important for education, but I am also a great believer in leveraging technology to augment traditional methods of education and to set a level playing field for all kids.
Let me know if you need more details, or want to send a stinker down my way. You can reach out via LinkedIn for that 🤣 !!
Thank you, and namaskara 🙏 !!
