.\" Automatically generated by Pod::Man 2.27 (Pod::Simple 3.28)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{
.    if \nF \{
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "Locale::Maketext::Utils::Phrase::Norm::NonBytesStr 3"
.TH Locale::Maketext::Utils::Phrase::Norm::NonBytesStr 3 "2014-11-18" "perl v5.16.3" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "Normalization"
.IX Header "Normalization"
We only want bytes strings and not “wide” unicode code point notation.
.SS "Rationale"
.IX Subsection "Rationale"
This helps give consistency, clarity, and simplicity.
.IP "\(bu" 4
Having one standard means no one has to guess/lookup what it is they are looking at or how they are expected to do it.
.IP "\(bu" 4
When harvesting phrases we avoid having to deal with interpolating in order to get the correct key to look up.
.Sp
Text::Extract::MaketextCallPhrases will handle it correctly for perl notation but what if you’re not parsing perl code?
.IP "\(bu" 4
At run time we avoid potential key to look up problems.
.IP "\(bu" 4
Avoids many encoding/decoding issue complexities.
.IP "\(bu" 4
Using unicode code point notation adds a layer of complexity that hinders translators and thus makes room for lower quality translations.
.IP "\(bu" 4
In perl, there's no really good way to combine the use of bytes strings and unicode string without issues. If we use bytes strings everything just works.
.Sp
Of course, using unicode strings when you need to operate under character semantics is the appropriate thing to do and newer perls have really great tools for that.
.Sp
However, for localization we are essentially looking up and passing through without examination or collation modifications. So bytes is the way to go for phrases!
.IP "\(bu" 4
Many things you might want to do with a phrase require it be bytes.
.Sp
You get garbled data when output to browser, file, database, or terminal.
.Sp
Various hashing and encrypting operate on bytes (using a unicode string can be fatal or you silently get unexpected data).
.PP
Solution: You can simply use the character itself or a bracket notation method for the handful of markup related or visually special characters
.SH "possible violations"
.IX Header "possible violations"
If you get false positives then that only goes to help highlight how ambiguity adds to the reason to avoid non-bytes strings!
.PP
Note that \s-1HTML\s0 Entities are not addressed here since the unicode notation as well as other syntax is covered via Ampersand.
.IP "non-bytes string (perl)'" 4
.IX Item "non-bytes string (perl)'"
This means you have something like \ex{\s-1NNNN\s0} and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,non bytes unicode string “\ex{\s-1NNNN\s0}”]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.IP "charnames.pm string notation" 4
.IX Item "charnames.pm string notation"
This means you have something like \eN{…} and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,charnames.pm type string “\eN{…}”]’ so you can find them visually.
.IP "unicode code point notation (C/\*(C+/Java style)'" 4
.IX Item "unicode code point notation (C//Java style)'"
This means you have something like \euNNNN and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,unicode notation “\euNNNN”]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.IP "unicode code point notation (alternate style)" 4
.IX Item "unicode code point notation (alternate style)"
This means you have something like U'\s-1NNNN\s0' and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,unicode notation “U'\s-1NNNN\s0'”]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.IP "unicode code point notation (visual notation style)'" 4
.IX Item "unicode code point notation (visual notation style)'"
This means you have something like U+NNNN and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,non bytes unicode string “U+NNNN]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.IP "unicode code point notation (visual notation type 2 style)'" 4
.IX Item "unicode code point notation (visual notation type 2 style)'"
This means you have something like UxNNNN and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,non bytes unicode string “UxNNNN]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.IP "unicode code point notation (Python style)" 4
.IX Item "unicode code point notation (Python style)"
This means you have something like u\*(L"\euNNNN\*(R" and need to use the character itself instead.
.Sp
These will be turned into ‘[comment,non bytes unicode string “u\*(L"\euNNNN\*(R"”]’ (where \s-1NNNN\s0 is the Unicode code point) so you can find them visually.
.SH "possible warnings"
.IX Header "possible warnings"
None