{"id":71135,"date":"2024-05-23T18:00:36","date_gmt":"2024-05-23T22:00:36","guid":{"rendered":"https:\/\/news.samsung.com\/us\/?p=71135"},"modified":"2024-06-13T14:52:04","modified_gmt":"2024-06-13T18:52:04","slug":"the-learning-curve-part-3-taking-ai-data-from-good-to-great","status":"publish","type":"post","link":"https:\/\/news.samsung.com\/us\/samsung-galaxy-ai-recognizes-translates-vietnamese-language-learning-curve-3\/","title":{"rendered":"The Learning Curve, Part 3: Taking AI Data From Good to Great"},"content":{"rendered":"

Samsung is pioneering premium mobile AI experiences<\/a>. To learn how Galaxy AI<\/a>1<\/a><\/sup> is maximizing the potential of its users, we are visiting Samsung Research centers around the world<\/a>. Now supporting 16 languages, Galaxy AI is enabling more people to expand their language capabilities, even when offline, thanks to on-device translation in features<\/a> such as Live Translate<\/a>2<\/a>, Interpreter, Note Assist and Browsing Assist. We recently visited Jordan<\/a> to learn the complexities of developing an AI model for Arabic, a language with many dialects. This time, we\u2019re going to Vietnam to explore how data is prepared to train AI models.<\/p>\n

What is the difference between a ghost, grave and mother in Vietnamese? For a language spoken by 97 million people worldwide, very little. Each word translates to \u201cma,\u201d \u201cm\u1ea3\u201d and \u201cm\u00e1,\u201d respectively \u2014 and can only be distinguished by tone. This illustrates how difficult it can be for AI models to learn a language, considering they cannot recognize firsthand the context and emotions of conversations nor the intentions of those speaking.<\/p>\n

Samsung R&D Institute Vietnam (SRV) used finely refined data to help its AI model properly recognize even the most subtle differences in language.<\/p>\n

The quality of data used directly affects the accuracy of automatic speech recognition (ASR), neural machine translation (NMT) and text-to-speech (TTS) \u2014 processes that help Galaxy AI features such as Live Translate, Interpreter, Chat Assist and Browsing Assist break down language barriers.<\/p>\n

A Typhoon of Challenges<\/strong><\/h5>\n

\u201cVietnamese is a complex and diverse language with rich expressions, many of which are challenging to capture,\u201d says Ng\u00f4 H\u1ed3ng Th\u00e1i, NMT lead at SRV. Of the 16 languages<\/a> that Galaxy AI supports, Vietnamese was particularly difficult to develop.<\/p>\n

\"Group<\/p>\n

\u201cPersonally, creating an AI model for Vietnamese was more daunting than our typhoons!\u201d he adds before explaining the hurdles faced during the development process.<\/p>\n

\"chart<\/p>\n

Vietnamese is a tonal language with six distinct tones. As evident in the \u201cma\u201d example above, small nuances in vocalization can drastically alter the meanings of words. Therefore, a meticulous and detailed approach was necessary.<\/p>\n

\u201cWhen similar sounding words are broken down, one word consists of several short segments, or \u2018frame sets\u2019,\u201d says Bui Ngoc Tung, ASR lead at SRV. \u201cThe AI model differentiates between the short audio frames of around 20 milliseconds to recognize what words correspond to a certain set of consecutive frames. As such, it is critical to put great effort into the early stages of the AI learning process.\u201d<\/p>\n

\"Samsung<\/p>\n

Furthermore, homophones and homonyms are common in Vietnamese. People can normally rely on context and nonverbal elements in conversations to differentiate between words that sound the same or are written the same but have different meanings. However, AI models need to be taught to accurately identify and differentiate between tones and similar words.<\/p>\n

\u201cThis isn\u2019t a straightforward task,\u201d Th\u00e1i explains. \u201cApart from the amount, the data needs to be accurate to ensure it is capable of recognizing the linguistic nuances that exist in Vietnamese.\u201d<\/p>\n

\"back<\/p>\n

Rigorous Pr<\/strong>eparation<\/strong><\/h5>\n

The data refinement process consists of three steps. First, the audio and text used to train the AI model must be reviewed and corrected. Then, this dataset goes through random checks for overall quality. Finally, the dataset is normalized and cleaned before use in training.<\/p>\n

\"Steps<\/p>\n

\u201cWe thoroughly performed a series of tests to check the accuracy of our dataset,\u201d says Nguyen Manh Duy, TTS lead at SRV who oversees database creation. \u201cWe faced a number of unexpected problems including misspelled words in scripts and background noise or incorrect pronunciation during audio recordings. We spent significant time refining and improving our training data.\u201d<\/p>\n

\"Samsung<\/p>\n

In addition to the unique linguistic challenges in Vietnamese, there is a lack of universally accessible data compared to more widely spoken languages. \u201cThis is another reason why the data refinement stage is so important,\u201d he adds. \u201cSince we had limited sources, every piece of data had to be fully reliable. There was no margin for error.\u201d<\/p>\n

\"Meeting<\/p>\n

Moreover, the AI model for Vietnamese must consider both tonal and regional differences. To improve the AI model\u2019s accuracy, the team collected vast amounts of data with Vietnam\u2019s northern, central and southern accents \u2014 resulting in an enormous amount of information to refine and verify.<\/p>\n

Continued Improvement<\/strong><\/h5>\n

Developers at SRV completed the project after months of hard work, and Vietnamese became one of the first languages to be supported by Galaxy AI. Despite this success, the team is ceaselessly working to improve the Vietnamese Galaxy AI experience<\/a>.<\/p>\n\n\t\t<\/div>\n\t\t<\/div>\n\t\t

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Recommended News<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t\t\n\t\t\t\t\t\t\"The\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t

Smartphones<\/p>\n\t\t\t\t\t\t\t\t

The Learning Curve, Part 2: How to Build an AI for Diverse Dialects<\/h4>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t\t
\n\t\t
\n

\u201cWe\u2019re continuing to enhance the AI model by incorporating user feedback about the relevance of words and phrases in Galaxy AI<\/a>,\u201d says Tran Tuan Minh, leader of the AI language development project at SRV. \u201cWe have just taken our first steps into a more open world \u00a0\u2014\u00a0 and we have so much more to explore together.\u201d<\/p>\n

\"Meeting<\/p>\n

In the next episode of The Learning Curve, we will head to China to dig into how AI models are trained and fine-tuned.<\/em><\/p>\n

1<\/sup><\/a> Galaxy AI features by Samsung will be provided for free until the end of 2025 on supported Samsung Galaxy devices.<\/h6>\n
2<\/sup><\/a> Samsung account log-in required. Calls must be made using the native Samsung phone app. Samsung does not make any promises, assurances or guarantees as to the accuracy, completeness or reliability of the output provided by AI features.<\/h6>\n","protected":false},"excerpt":{"rendered":"

Samsung Research in Vietnam is part of a series about the people and innovations allowing mobile AI to enhance more lives.<\/p>\n","protected":false},"author":84,"featured_media":71141,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[29720,29721],"tags":[953,25940,30339,30338,40,30424,30425],"blue-badge":[],"class_list":["post-71135","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-product-mobile","category-product-mobile-smartphones","tag-artificial-intelligence-ai","tag-galaxy","tag-galaxy-ai","tag-live-translate","tag-mobile","tag-samsung-rd","tag-the-learning-curve"],"acf":{"turn_off_retargeting":false},"fimg_mobile_url":"https:\/\/img.us.news.samsung.com\/us\/wp-content\/uploads\/2024\/05\/23144703\/samsung-vietnam-research-team-200x200.jpg","fimg_url":"https:\/\/img.us.news.samsung.com\/us\/wp-content\/uploads\/2024\/05\/23144703\/samsung-vietnam-research-team-432x286.jpg","primary_category":{"term_id":29721,"name":"Smartphones","slug":"product-mobile-smartphones","term_group":0,"term_taxonomy_id":29721,"taxonomy":"category","description":"","parent":29720,"count":401,"filter":"raw","term_link":"https:\/\/news.samsung.com\/us\/category\/product\/product-mobile\/product-mobile-smartphones\/","term_path":"product\/product-mobile\/product-mobile-smartphones"},"badge":false,"_links":{"self":[{"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/posts\/71135","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/users\/84"}],"replies":[{"embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/comments?post=71135"}],"version-history":[{"count":0,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/posts\/71135\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/media\/71141"}],"wp:attachment":[{"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/media?parent=71135"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/categories?post=71135"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/tags?post=71135"},{"taxonomy":"blue-badge","embeddable":true,"href":"https:\/\/news.samsung.com\/us\/wp-json\/wp\/v2\/blue-badge?post=71135"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}