60 lines
8.2 KiB
Plaintext
60 lines
8.2 KiB
Plaintext
<h2 id="telegram-messengers-newest-update">Telegram Messenger’s newest update</h2>
|
||
|
||
<p>You might have heard that Telegram has released arguably <a href="https://telegram.org/blog/reactions-spoilers-translations">their biggest update of the year</a> this week. While the backend of the messaging platform remains proprietary, the source code of the mobile and desktop clients is<a href="https://github.com/DrKLO/Telegram"> open source</a>.</p>
|
||
|
||
<p>The big new feature is <a href="https://telegram.org/blog/reactions-spoilers-translations#message-translation">Message Translations</a>, which allows to translate the text of messages within the app. What is interesting is how this is implemented in the official Android app.</p>
|
||
|
||
<h2 id="how-the-telegram-android-app-circumvents-the-official-google-cloud-translate-api">How the Telegram Android app circumvents the official Google Cloud Translate API</h2>
|
||
|
||
<h3 id="undocumented-google-translate-api-endpoint">Undocumented Google Translate API endpoint</h3>
|
||
|
||
<p>If you check the official <a href="https://cloud.google.com/translate/docs/reference/rest/v2/translate">Cloud Translate REST API documentation</a>, you will see that the official API uses a versioned API path (e.g. <code class="language-plaintext highlighter-rouge">/language/translate/v2</code>), and human readable query parameters, which importantly include the API key <code class="language-plaintext highlighter-rouge">key</code>. However, if we check<a href="https://github.com/DrKLO/Telegram/commit/9e740dfd4d2b1ab6b8ed2b972e0f72fc9b8bd09d#diff-bc405602f072ccdb4e595ac9f577f6bfae72778c6a989bf81021b79cfef28568R1081"> Telegram’s implementation</a>, we will notice a few things in the <code class="language-plaintext highlighter-rouge">fetchTranslation</code> method:</p>
|
||
|
||
<p>They use another path, and also seem to intentionally split up the request path with multiple string joins (perhaps for obscurity / avoid detection in the Play Store review process?):</p>
|
||
|
||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">uri</span> <span class="o">=</span> <span class="s">"https://translate.goo"</span><span class="o">;</span>
|
||
<span class="n">uri</span> <span class="o">+=</span> <span class="s">"gleapis.com/transl"</span><span class="o">;</span>
|
||
<span class="n">uri</span> <span class="o">+=</span> <span class="s">"ate_a"</span><span class="o">;</span>
|
||
<span class="n">uri</span> <span class="o">+=</span> <span class="s">"/singl"</span><span class="o">;</span>
|
||
<span class="n">uri</span> <span class="o">+=</span> <span class="s">"e?client=gtx&sl="</span> <span class="o">+</span> <span class="nc">Uri</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">fromLanguage</span><span class="o">)</span> <span class="o">+</span> <span class="s">"&tl="</span> <span class="o">+</span> <span class="nc">Uri</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">toLanguage</span><span class="o">)</span> <span class="o">+</span> <span class="s">"&dt=t"</span> <span class="o">+</span> <span class="s">"&ie=UTF-8&oe=UTF-8&otf=1&ssel=0&tsel=0&kc=7&dt=at&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&q="</span><span class="o">;</span>
|
||
<span class="n">uri</span> <span class="o">+=</span> <span class="nc">Uri</span><span class="o">.</span><span class="na">encode</span><span class="o">(</span><span class="n">text</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>We can deduce from the query string that:</p>
|
||
|
||
<ul>
|
||
<li><code class="language-plaintext highlighter-rouge">client</code> is some kind of client caller specifier (e.g. webapp / native app?)</li>
|
||
<li><code class="language-plaintext highlighter-rouge">sl</code> and <code class="language-plaintext highlighter-rouge">tl</code> are source and target languages</li>
|
||
<li><code class="language-plaintext highlighter-rouge">ie</code> and <code class="language-plaintext highlighter-rouge">oe</code> are input and output encoding of the text data</li>
|
||
<li><code class="language-plaintext highlighter-rouge">ssel</code> and <code class="language-plaintext highlighter-rouge">tsel</code> have something to do with text selection?</li>
|
||
<li><code class="language-plaintext highlighter-rouge">q</code> is the query text (the URI encoded text to actually translate)</li>
|
||
</ul>
|
||
|
||
<p><strong>UPDATE</strong>: This workaround is explained very well in <a href="https://vielhuber.de/en/blog/google-translation-api-hacking/">this blog post</a>, so definitely check it out.</p>
|
||
|
||
<h3 id="user-agent-rotation">User agent rotation</h3>
|
||
|
||
<p>Another thing I noticed is that Telegram keeps an array of strings containing various User Agents, with comments indicating percentages (what they represent is not clear to me at the moment):</p>
|
||
|
||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">private</span> <span class="nc">String</span><span class="o">[]</span> <span class="n">userAgents</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">String</span><span class="o">[]</span> <span class="o">{</span>
|
||
<span class="s">"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"</span><span class="o">,</span> <span class="c1">// 13.5%</span>
|
||
<span class="s">"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"</span><span class="o">,</span> <span class="c1">// 6.6%</span>
|
||
<span class="s">"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0"</span><span class="o">,</span> <span class="c1">// 6.4%</span>
|
||
<span class="s">"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0"</span><span class="o">,</span> <span class="c1">// 6.2%</span>
|
||
<span class="s">"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36"</span><span class="o">,</span> <span class="c1">// 5.2%</span>
|
||
<span class="s">"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36"</span> <span class="c1">// 4.8%</span>
|
||
<span class="o">};</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>In the same method, it seems that they randomly pull a user agent from this array and pass it to the request to Google:</p>
|
||
|
||
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">connection</span><span class="o">.</span><span class="na">setRequestProperty</span><span class="o">(</span><span class="s">"User-Agent"</span><span class="o">,</span> <span class="n">userAgents</span><span class="o">[(</span><span class="kt">int</span><span class="o">)</span> <span class="nc">Math</span><span class="o">.</span><span class="na">round</span><span class="o">(</span><span class="nc">Math</span><span class="o">.</span><span class="na">random</span><span class="o">()</span> <span class="o">*</span> <span class="o">(</span><span class="n">userAgents</span><span class="o">.</span><span class="na">length</span> <span class="o">-</span> <span class="mi">1</span><span class="o">))]);</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>It seems like a classic example of user agent rotation, a technique often used by web scrapers to avoid being rate limited / blacklisted by web services.</p>
|
||
|
||
<h2 id="conclusion">Conclusion</h2>
|
||
|
||
<p>It seems that to get around the problem of translating text on Android in Telegram and not pay huge Google Cloud fees and risk leaking their API key, Telegram found some obscure way of querying the Cloud Translate API directly at no cost to them.</p>
|
||
|
||
<p>My advice would be to simply use their pre-built official Java SDK, and utilize <a href="https://cloud.google.com/translate/docs/reference/rpc">RPC over HTTP</a> to save on bandwidth (which will be very substantial given Telegram’s <a href="https://t.me/durov/142">over 500 million active users</a>. To me it seems the feature was heavily rushed in time for the end of the year, given the state of the new code linked above.</p> |