Ask HN: Conversational AI to Learn a Language
Hi all,
I want to learn Dutch and by experience I know I learn better when talking with native speakers.
From your experience, is there a good AI I can converse with in Dutch? It would be even better if I could see the transcription in Dutch.
Gliglish (https://gliglish.com) looks good but is more for speaking than for learning. I'd like to be able to set a situation (negotiating a job offer, calling a supplier ...).
Hi! I have a WIP of this over at https://talktrainer.app/ -- I just added Dutch to it.
It uses OpenAI's realtime API to simulate either a tutoring session (the speaker will revert to English to help you) or a first date or business meeting (the speaker will always speak the target language)
You can see the AI's transcriptions but not your own, limitation of the current OpenAI API but definitely something I can fix.
The prompts are like this: https://gist.github.com/jc4p/d8b9d121425ec191d62602d8720eeed... and the rest of it is a Nextjs app wrapped around the WebRTC connection.
I'm not fully in love with the app so I'd love any feedback or hearing if it works well for you -- It doesn't have a lot of features yet (including saving context) and if you bump into the time limit just open it up in incognito to keep going.
This is great! Maybe some more tourist-related scenarios, like "ordering at restaurant", "resolving dispute about rental car crash" etc? :-)
The "next level" feature would be to get it to speak even simpler, with some hints about how to reply, for the beginners. I don't know how that would ideally look, but maybe a button to pop up some "key words" or phrases that one could use? (Even so, I found myself using the little I know, so it's obviously somehow working even though my knowledge is extremely basic.)
This is one of the places where I feel LLM's can do something good for the world, giving a safe playground for getting experience with speaking new languages without the anxiety of performing badly in front of other people – and hopefully make it easier to connect with real people in that language later.
This is really impressive! Great job.
One small piece of feedback… There were a couple times where I asked to learn something, and it asked me to repeat a phrase back, which was great. But when I repeated it back, I know I didn’t quite nail it (eg perhaps said “un” instead of “una”) and rather than correcting me, it actually told me I did it perfectly. Maybe there’s some tuning with the prompts that may help turn down the natural sycophancy of the model and make sure it’s a little more strict.
Keep up the great work!
Did you just add Dutch as per the submitter’s request or was it part of your plan prior?
Curious because I’m trying to learn Romanian, and since it’s a less common language there are fewer resources available. So I wasn’t sure if you added Dutch with minimal amount of effort following the poster’s request.
That said, I gave your app a try with Spanish and it looks pretty good! But I didn’t see a Help page to clarify how I’m “supposed” to interact. Eg I tried saying in English “I don’t understand” (even though I know how to say that in Spanish) and it responded in Spanish which may be hard for absolute beginners. Although full immersion is much better way to learn.
I can try playing around more with it to give you some feedback.
> Eg I tried saying in English “I don’t understand” (even though I know how to say that in Spanish) and it responded in Spanish which may be hard for absolute beginners.
I tried to use ChatGPT as a "live" translator with my in laws and I noticed it is extremely bad at language "consistency" or at understanding your intent when it comes to multiple languages.
It will sometimes respond in English when you talk to it in the foreign language, it will sometimes assume that a clear instruction like "repeat the last sentence" needs to be translated, etc.
I don't know how the person above is approaching the problem but your experience is consistent with mine and I don't think GenAI models (at least OpenAI ones) are suitable for the task.
I just tried it and it works perfectly. The color scheme and font size could be touched up to look better. Just out of curiosity, is $10/month enough to cover the (unlimited) API cost? Do you estimate how many percentage of your users will use more than $10 API fee each month?
Thanks so much for trying it out! The realtime API is actually very cheap especially for short connections, for each user who uses it 30 minutes a day every day in a month it costs me ~$5 and I assume the average user is going to use it way less than that (although i have 0 users right now haha)
This is great! Well done.
I've used the realtime API for something similar (also related to practicing speaking, though not for foreign languages). I just wanted to comment that the realtime API will definitely give you the user's transcriptions -- they come back as an `server.conversation.item.input_audio_transcription.completed` event. I use it in my app for exactly that purpose.
Thank you so much!! While the transcription is technically in the API it's not a native part of the model and runs through Whisper separately, in my testing with it I often end up with a transcription that's a different language than what the user is speaking and the current API has no way to force a language on the internal Whisper call.
If the language is correct, a lot of the times the exact text isn't 100% accurate, if that's 100% accurate, it comes in slower than the audio output and not in real time. All in all not what I would consider feature ready to release in my app.
What I've been thinking about is switching to a full audio in --> transcribe --> send to LLM --> TTS pipeline, in which case I would be able to show the exact input to the model, but that's way more work than just one single OpenAI API call.
Ah yes, I've seen that occasionally too, but it hasn't been a big enough issue for me to block adoption in a non-productized tool.
I actually implemented the STT -> LLM -> TTS pipeline, too, and I allow users to switch between them. It's far less interactive, but it also gives much higher quality responses.
Best of luck!
[dead]
If you have a ChatGPT subscription, set up your own GPT with prompting around your level, how you want it to respond, how to correct mistakes etc. Then you can use it for anything - Generate tests based on words you know, roleplay like ordering in a restaurant, write stories and have it correct grammar.
This is what I have to supplement my Chinese and it is incredibly helpful.
Look at the comments already - Everyone is building a simple wrapper to do this very thing but charge you $20 per month for the privelege. These are souless, most likely vibe coded garbage. Avoid.
You don't even need a subscription, just start the conversation with something like "I'm trying to learn x, I'm at a beginner skill level. Can you have a conversation with me and correct my mistakes." and it works superbly.
The Duolingo CEO copped a lot of criticism for it, but I think he is right that LLMs play a huge role in the future of education, though he probably ragebaited everyone by overselling it and calling teachers babysitters. But ChatGPT as it is now is a better language learning tool than their hand crafted app is. Rather than clicking on word blocks, you can actually have a free form conversation and get feedback like "Yes your sentence is understandable but sounds unnatural, you could try ___"
Duolingo CEO just went on record to say that all future language teaching will be AI, but physical schools and teachers will continue to exist "to provide childcare".
That is a very polarizing way to phrase it.
Luckily for teachers and schools in general, (foreign) language skills are just a small part of overall school curriculum and the higher you go the more this becomes true. LLMs not taking that part away anytime soon.
You can also share them, e.g. someone made a specialised ChatGPT for korean:
https://chatgpt.com/g/g-erkTp2LNZ-learn-korean-with-gpt
I'm not sure how it works though. Just a canned prompt?
I want to practice speaking and last time I checked Advanced Voice Mode was not available in custom GPTs - is that still the case? Advanced Voice made a really big difference to conversational practice for me, but I do feel like I need to wrap it in a custom knowledge base/instructions to make it as useful as it could be for my language practice.
For the LLMs uninitiated - I checked and ChatGPT Plus costs 20$/month, which puts it into same category as those services others build. Is this what you meant by having a subscription?
If I don't use ChatGPT for other purposes it seems like same prices to me, without the hassle of setting it up and tweaking. Or am I missing something?
A lot of HN users already have subscriptions to one of these LLM services.
I am working on a telegram bot for this https://t.me/FriendFluentBot - It doesn't support dutch yet but i can add it.
I am a pakistani living in germany and married into a turkish family.The main purpose for making this is to help myself and family to communicate with each other. I still haven't landed on a magic approach to learn language in 7 days. But this bot is for having human like conversations about anything. It'll remember your past history as well as what you have already talked about to keep the natural flow.
Audio messages are still a work in progress. Will be added by the end of the day.
Any feedback would be appreciated as it is still in very early stages.
I've been using Univerbal happily with Italian. Dutch is one of its 20 languages. Worth noting that it's a paid app but seems to have a lot of polish. I found out about it on HN so I'm sure you can look up discussions to get more opinions on it (though it's quite frequently updated).
https://www.univerbal.app
I use ChatGPT's conversation mode to supplement my language learning (A2/B1-ish French and 1hr/wk 1-on-1 tutor). I tend to use it in the car, just asking about random facts or ideas or playing 20-questions.
The format forces me to just use my voice and listening skills - in other words, I'm forced to not touch my phone. It's also rather challenging because I'm doing two things at once and the hope is that I won't actually spend much brain power overthinking my responses - something I tend to do if I was talking to myself instead which typically turns into more of a rehearsal format.
I’m building https://instantlyfluent.com, a small project to help people and myself practice languages before travel. It works through voice or text chats with AI.
I tried Gliglish but didn’t like how it was structured. So I started turning my own “learn a language before a trip” routine into something easier to use.
Dutch is not supported yet, and there are still some rough edges. But the idea is simple: help people remember words by using them. You say or type something, and get a reply you can hear or read.
In conversations on IF, there is no fixed path. You can start with a topic, then ask to switch to something else. It’s meant to be low-pressure and flexible.
Still figuring things out. Happy to share more or hear from others working on similar tools.
Our Voice AI Agent makes it extremely simple and engaging to learn new language - create card dynamically, asks multiple-choice-questions, uses diagrams, does role play ...
Demo: https://www.youtube.com/watch?v=2iSIVnLR-nM Website: https://app.toughtongueai.com/
Why is video 1.5x accelerated?
I'm trying to solve the problem of unlimited conversation practice for various languages with "CallAnnie", an iOS/android app with various kind of AI friends, powered by real time voice and avatars.
The interface is specifically made for advanced learners that want to simulate a conversation as close as possible to a real one (in terms of latency and without pushing any button). Learning to respond fast in a new language is important, so we're trying to keep a natural pace.
We support audio or video-calling the characters (with subtitle translations), guided conversations and we recently added mini games to learn vocabulary.
You can see a quick intro and demo video here: https://www.sixthtone.com/news/1013961
We don't have a Dutch speaking character in the default character list, but you can follow this link https://app.callannie.ai/a/mhLnHflAyf1Ygb0D0wm6 after installing the app to use a custom character speaking Dutch (or you can create a custom character).
If you'd like to try it, check out: callannie.ai . I would love to get your feedback (here or francesco a t callannie.ai) and suggestions - we're trying to solve this speaking practice use case.
@jeffwass why would anyone want to learn Romanian ..no but seriously buckle up because it's the most irregular language ever with weird expressions and when you ask why something is gramatically how it is you'll hear "idk it just sounds better" so get ready for that
I'm learning Finnish. For the most part, Gemini and ChatGPT do a much better job at generating passable if sightly Internet-inflected Finnish than I'm likely to anyway within the next 5 years, so I just talk with those.
I would imagine Dutch to be in the same camp, unless a native Dutch speaker reviews some of your conversations and tells you otherwise. My native Finnish wife has given me the marginal all-clear with the vanilla models.
What are some of the prompts you use?
I tried Spanish (I don't know Dutch at all) with Claude via duck.ai.
I immediately had to correct it. (My Spanish is limited to a half a semester 20 something years ago and season 1 of Community.) I don't think I'd be confident that I'd learn accurately from an AI.
I've been setting up Vapi.ai for this. Written and spoken are different skills. It comes down cheaper than an actual teacher, and it has the patience to correct you.
NotebookLM now supports Dutch - So you could load up some content of interest and 'join the studio' to talk with the hosts - I imagine that might be useful and fun.
I added a Practice Mode in my speech-to-speech translator app 3PO. It allows you to practise talking with chatGPT and gets assessment score on your accuracy.
Topic is preset. (chatGPT plays the role of a travel guide to help you find places to go in Holland (for Dutch))
https://3po.evergreen-labs.org
Dope name. Beware of Disney though.
I didn't try to learn Dutch. But I have used AI for learning Spanish. From my experience, they can guide and teach you the "School Spanish" perfectly. However, they cannot teach you slangs and conversational Spanish.
Try https://www.issen.com (YC company).
Any of those "AI companion" apps can do this very well, or so I have read!
I’m making fluent.im, it has situations with goals for each conversation, and we’re about to ship lessons based on your favorite podcast content.
Im not sure in Dutch... But recommends starting with chatgpt until find a good product.
> I want to learn Dutch and by experience I know I learn better when talking with native speakers.
If you are learning Dutch to talk to native speakers then why don't you talk to native speakers. You clearly plan to find actual people to talk to at some point right? So go ahead and do it
You will improve your language skills this way, but if your skill level isn't high enough it's going to be exhausting to the native speaker as well.
I have been living in Denmark for 15 years now, and it's still easier to do conversations in English. When I speak Danish it requires more mental capacity from the other side.
I am speaking Danish from time to time, but it's only to get better at it. The English proficiency in Denmark (and probably the Netherlands) is so high that you need to be really good at the native tongue before it is easier than English in conversations.
It helps if you pay natives for their suffering. Something like Italku does just that.