Hi Stas,
On Mon, Feb 17, 2014 at 12:13 PM, Stas Malyshev <[email protected]>wrote:
> > operation. There are a ton of non-obvious things beyond simple string
> > manipulation. String collation alone is massively complicated, for
> example.
>
> Oh yes, and if somebody thinks case sensitivity is weird now, wait until
> Unicode gets into play. There for some chars when you change the case
> string length changes, and for some conversion is not roundtrip-safe.
> And you have various long form/short form combining issues which means
> you need to normalize everything on every corner. So letting Unicode
> into things like identifiers opens a huge container of worms.
> Also, if one wants to appreciate what other cans of worms are hiding
> there, I recommend this oldie but goodie:
> http://stackoverflow.com/a/6163129/214196
> It's about Perl, but we'd have many of the same issues.
Nice article. I mostly agree.
"Code that converts unknown characters to ? is broken, stupid, braindead,
and runs contrary to the standard recommendation, which says NOT TO DO
THAT! RTFM for why not."
While I agree this (It's BAD to accept broken text as valid input), there
are situations that programmer has to handle broken text. Ruby finally
admits scrab method is needed. It's available from Ruby 2.1.0.
Regards,
--
Yasuo Ohgaki
[email protected]