I18n

[yorkxin]

Hi,

We're going to host a workshop in Taiwan, which uses RailsBridge materials. Obviously it should be translated into Traditional Chinese, the language we use in Taiwan.

During translation I found that there are some words that cannot be translated until I18n is applied. So I implemented this.

After this PR I will send another PR which adds zh-tw locale into app.rb and the flag of Taiwan at the top-right corner.

By the way, we've done translating "Installfest" and "Intro to Rails" sections. Here is the URL: http://railsbridge-docs-zh-tw.herokuapp.com/docs/

---

Update: further details:

For common words, like "Next Step", "Step X", they're going to be localized in locales/<locale>.yml file.

For page names, currently English is automatically generated from file name, like what it does currently. For other languages, page names should be translated in locales/pages.<locale>.yml file. If the translation is missing, it will take the auto-generate strategy, which fallbacks to English. There is a locales/pages.en.yml file for those who're going to localize page names.

[alexch]

Excellent! Can you please add your name to this google doc so we can
discuss translation issues with you? We'd love to learn from your
experience and make it easier for you to translate more pages.

https://docs.google.com/document/d/1bi1F8B4x2zh_ICNGMp1gE30y-Q-d-rdQK84My3dTCe4/edit?usp=sharing

[tjgrathwell]

Hi @chitsaou, this looks great to me as far as the I18n support for helpers like goals, next_step etc.

But I question the need to have a separate locale file for translating page names. For example, when the site was translated into Spanish, the site and page names were translated as well, as you can see on pages like http://es.railsbridge.org/introduccion-a-rails/creditos_y_pasos_siguientes. This provides the added benefit of having URLs in the translated language. Can you explain your rationale for not wanting to do that so I can better understand where you're going with this?

Also, it would be nice for commits like fix i18n-related tests (1162173) to be squashed into whatever commit broke the tests: ideally, the tests should be passing after every individual commit.

[alexch]

This change seems backwards to me. Why change the name of a perfectly good method and then stop using it, in favor of some static global?

[yorkxin]

The document I followed was Sinatra I18n guide. I18n, as a global module, holds the current locale in the environment, so that I18n.t and other methods work. It is per-request (wrapped in before block) so it should not be so static that affects other requests.

[yorkxin]

I also wondered if changing locale to get_locale is good or not. Keeping it locale may confuse with I18n.locale, but get_locale is not a Ruby-ish naming.

Suggestions?

[alexch]

I need to check into the status of the Spanish fork... It really needs to be merged before this PR to make sure we got all the necessary settings.

[yorkxin]

Hi @tjgrathwell,
The reason I did not translated URLs is that I tend not to put Chinese (or non-latin chars) into URLs.

In Taiwan we actually don't like to use Chinese characters in URLs, not only because typing Chinese in address bar is annoying (English is much easier to type and everyone understands very basic English), but also because that, when copying a Chinese URL and paste it to elsewhere, different browser implements different behaviors. For example in Chrome it copies URL-Encoded strings (sequences of "%xx%yy" which is nonsense to general people), but Safari copies the URL in Chinese as-is, which may not be recognized as a part of URL in some software.

Also, I feel it not so good to compute page name from its file name, which may introduce more special transforming cases in the future and break other locales.

However I haven't tried auto-generation strategy in Chinese. I'll try it and tell you whether it actually fits the need or not. If everything is OK, I'll discard my new pages.yml strategy.

[tjgrathwell]

That's interesting to hear @chitsaou. I don't have much experience in I18n or knowledge of how widespread non-latin characters are in URLs. My main comparison for a site like the RailsBridge docs is http://zh.wikipedia.org which does use Chinese characters in the URL. But I don't know how common that is generally, and it does exhibit the wacky copying behavior you describe.

It seems fine then if you want to do something like the pages.zh.yml solution. Though I think instead of checking in a pages.en.yml "template" which may become out of date it might be better to just check in a script that generates that content, which could be used when development on a new translated site starts. Also the underscore-prefixed partials like _install_rvm can be omitted because they never show in the URL or page titles.

@alexch what kind of stuff needs to be merged from the Spanish fork before this can get in?

[yorkxin]

@tjgrathwell Good suggestion. It is better to make a rake task (or something like that) to generate pages.yml from all view files, instead of hard-coded which may be outdated.

But first I'll try putting Chinese characters in the URLs according to your suggestion. As Wikipedia uses it, and I even have never noticed about that, maybe it is actually acceptable to people in Taiwan. It also solves the issue that "Back to " is untranslatable, as I noted in the comment. I'll discuss about this with my Taiwanese friends.

@alexch I think the only thing you have to add is es.yml which translates common words like "Next Step" into Spanish.

P.S. Maybe my implementation is not so good. If you encounter any problem or think there is a better way to do, feel free to tell me. I appreciate any suggestion. 😄

[ultrasaurus]

what did you all end up doing with the URLs? Interested in what @chitsaou found out about usability in Taiwan. I used Chinsese characters in Mightyverse, but regretted it since they look so bad in email and was thinking of switching back to ascii. Here's an example: http://www.mightyverse.com/en/phrases/Are+you+happy+with+the+current+situation%3F/zh-cmn/%E4%BD%A0%E5%AF%B9%E4%BD%A0%E7%9A%84%E7%8E%B0%E7%8A%B6%E6%BB%A1%E6%84%8F%E5%90%97%EF%BC%9F

[yorkxin]

Without this change, non-latin characters are not escaped and Ruby throws incompatible string encoding exception (US-ASCII v.s. UTF-8).

But with this change, some characters in URI::PATTERN::RESERVED are not escaped properly:

2.0.0-p451 :006 > URI.encode URI::PATTERN::RESERVED
 => ";/?:@&=+$,%5C[%5C]"
2.0.0-p451 :007 > URI.encode URI::PATTERN::RESERVED, URI::PATTERN::RESERVED
 => "%3B%2F%3F%3A%40%26%3D%2B%24%2C%5C%5B%5C%5D"

[yorkxin]

Also, with this change, the # char will be escaped, causing an invalid link when visiting this path:

Intro to Rails > Create Migration > Go on to: Rails Architecture > Go back: Create Migration

[yorkxin]

I've removed pages.yml implementation and now I've translated file names and URLs into Chinese too. This also solves the problem of untranslatable next_step labels, which looks better (visually) than pages.yml approach.

However I hit some issue in the "back to" link, specifically, when it contains URL fragment part. I've added notes on corresponding lines. I think this will break current English and Spanish website, so please don't Merge this until a better solution is found.

This is the only issue I found, and I'm not sure whether there are any other issues.

Yet another issue -- not pragmatically -- is that when I'm editing documents, I cannot quick-find file in Sublime Text with Command-T, because most files are now named in Chinese instead of latin characters. And I also have to make sure that all the link URLs are correct, which requires a carpet-searching.

Anyway I've deployed this to our production site, and see if we hit other bugs or people are arguing about the URL. (Though I don't think they'll find anything different.)

[tjgrathwell]

@alexch @chitsaou I'd love to get some of this merged in, are there any unresolved issues here? Other secret i18n work going on that needs to be merged in?

[yorkxin]

@tjgrathwell Sorry for late response. The only unresolved issue is the broken back link. I'll try to resolve this issue this week and let you know.

btw we don't have any secret code going on in Chinese version :p

[alexch]

"secret" no but I think there's still an outstanding Spanish fork. It
should be good to merge assuming we don't mind cluttering the version
history a tad. Just a smidge. A teeny tiny itty bitty bit.

On Mon, Jun 9, 2014 at 4:10 AM, Yu-Cheng Chuang [email protected]
wrote:

@tjgrathwell https://github.com/tjgrathwell Sorry for late response.
The only unresolved issue is the broken back link. I'll try to resolve this
issue this week and let you know.

btw we don't have any secret code going on in Chinese version :p

—
Reply to this email directly or view it on GitHub
#353 (comment).

Alex Chaffee - [email protected]
http://alexchaffee.com
http://twitter.com/alexch

[yorkxin]

I've resolve the issue when generating a back link that comes with fragment part. I think it is good to merge now!

[tjgrathwell]

Hi @chitsaou, I merged most of this code but made an amendment (01ca022) to bring back URI::PATTERN::RESERVED in the _escape function.

Your solution here causes a regression for the original issue that required me to add URI::PATTERN::RESERVED. The issue is that links with colons in them don't get escaped properly. You can see it if you visit a page like http://localhost:9292/ruby/running_programs_from_a_file and try to click on the "Go on to Summary: Tools" link on the bottom. If you leave out URI::PATTERN::RESERVED from the escape call the link doesn't work.

There's probably some version of URI.escape that will work for both the colon issue and non-ASCII characters, I just didn't work too hard to find it yet.

[yorkxin]

Hi @tjgrathwell,

Thank you for pointing it out. I didn't know that colon issue. Sorry for making a trouble.

So far I've found ERB::Utils.url_encode and CGI.escape. They both scan for non-basic-latin characters /[^a-zA-Z0-9_\-.]/ and escape them with %xx representation. The only difference is that CGI.escape encodes space (U+0020) into + sign, while ERB::Utils.url_encode encodes it as %20.

I think they're both better than using URI.encode which comes with a complex logic. I'll check if your changes break in Chinese version and let you know ERB or CGI versions solve the problem properly.