+
Of+) R t ^ RIt^RIHt RtRRRRRR R
R /t/ tR tR
tR t ! R R]
4 t ]! R4 t ]! R4 t
]! R4 tR R ltR t] R3R ltR R ltR t] R3R ltR t ! R R ]
4 t ! R R ]
4 tR# )!z
webencodings
~~~~~~~~~~~~
This is a Python implementation of the `WHATWG Encoding standard
`. See README for details.
:copyright: Copyright 2012 by Simon Sapin
:license: BSD, see LICENSE for details.
N)LABELSz0.6-devz iso-8859-8-iz
iso-8859-8zx-mac-cyrillicz mac-cyrillic macintoshz mac-romanz windows-874cp874c ^ V P R4 P 4 P R4 # )a Transform (only) ASCII letters to lower case: A-Z is mapped to a-z.
:param string: A Unicode string.
:returns: A new Unicode string.
This is used for `ASCII case-insensitive
`_
matching of encoding labels.
The same matching is also used, among other things,
for `CSS keywords `_.
This is different from the :meth:`~py:str.lower` method of Unicode strings
which also affect non-ASCII characters,
sometimes mapping them into the ASCII range:
>>> keyword = u'Bac\N{KELVIN SIGN}ground'
>>> assert keyword.lower() == u'background'
>>> assert ascii_lower(keyword) != keyword.lower()
>>> assert ascii_lower(keyword) == u'bac\N{KELVIN SIGN}ground'
utf8)encodelowerdecode)strings &H/usr/lib/python3/dist-packages/html5lib/contrib/webencodings/__init__.py ascii_lowerr ! s' . == & & ( / / 77 c > \ V P R4 4 p \ P ! V 4 pVf R# \ P V4 pVfN VR8X d ^RIHp M+\ P W4 p\ P ! V4 p\ W4 pV\ V& V# )u
Look for an encoding by its label.
This is the spec’s `get an encoding
`_ algorithm.
Supported labels are listed there.
:param label: A string.
:returns:
An :class:`Encoding` object, or :obj:`None` for an unknown label.
z
Nzx-user-defined)
codec_info) r stripr getCACHEx_user_definedr PYTHON_NAMEScodecslookupEncoding)labelnameencodingr python_names & r r r ; s
K0 1E ::e D |yyH # # 2&**46K {3JD- d Or
c p \ V R4 ' d V # \ V 4 pVf \ RV , 4 hV# )z
Accept either an encoding object or label.
:param encoding: An :class:`Encoding` object or a label string.
:returns: An :class:`Encoding` object.
:raises: :exc:`~exceptions.LookupError` for an unknown label.
r zUnknown encoding label: %r)hasattrr LookupError)encoding_or_labelr s & r
_get_encodingr Y s@ ,// '(H69JJKK Or
c 0 a ] tR t^kt o RtR tR tRtV tR# )r a/ Reresents a character encoding such as UTF-8,
that can be used for decoding or encoding.
.. attribute:: name
Canonical name of the encoding
.. attribute:: codec_info
The actual implementation of the encoding,
a stdlib :class:`~codecs.CodecInfo` object.
See :func:`codecs.register`.
c Wn W n R # N)r r )selfr r s &&&r __init__Encoding.__init__z s $r
c ( R V P , # )z
)r )r$ s &r __repr__Encoding.__repr__~ s **r
)r r N) __name__
__module__ __qualname____firstlineno____doc__r% r( __static_attributes____classdictcell__
__classdict__s @r r r k s
%+ +r
r zutf-8zutf-16lezutf-16bec \ V4 p\ V 4 w r0T;' g TpVP P W4 ^ , V3# )a
Decode a single string.
:param input: A byte string
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does note have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
:return:
A ``(output, encoding)`` tuple of an Unicode string
and an :obj:`Encoding`.
)r _detect_bomr r )inputfallback_encodingerrors bom_encodingr s &&& r r r sL &&78%e,L00 0H % %e 4Q 7 AAr
c V P R4 ' d \ V R, 3# V P R4 ' d \ V R, 3# V P R4 ' d \ V R, 3# RV 3# )zBReturn (bom_encoding, input), with any BOM removed from the input.s : NNs s : NNN)
startswith_UTF16LE_UTF16BEUTF8)r5 s &r r4 r4 sk $$r"" $$r"" ((U2Y ;r
strictc X \ V4 P P W4 ^ , # )a
Encode a single string.
:param input: An Unicode string.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
:return: A byte string.
)r r r )r5 r r7 s &&&r r r s% " - - 4 4U CA FFr
c J \ W4 p\ W4 p\ V4 pWE3# )a
"Pull"-based decoder.
:param input:
An iterable of byte strings.
The input is first consumed just enough to determine the encoding
based on the precense of a BOM,
then consumed on demand when the return value is.
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does note have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
:returns:
An ``(output, encoding)`` tuple.
:obj:`output` is an iterable of Unicode strings,
:obj:`encoding` is the :obj:`Encoding` that is being used.
)IncrementalDecoder_iter_decode_generatornext)r5 r6 r7 decoder generatorr s &&& r iter_decoderH s+ , !!2;G&u6IIH r
c # " VP p\ V 4 p V F7 pV! V4 pV' g K VP f Q hVP x Vx M8 V! RRR7 pVP f Q hVP x V' d Vx R# V F pV! V4 pV' g K Vx K V! RRR7 pV' d Vx R# R# 5i)ziReturn a generator that first yields the :obj:`Encoding`,
then yields output chukns as Unicode strings.
Nr
Tfinal)r iterr )r5 rF r chunckoutputs && r rD rD s
^^F KE 6##/ //"" "L
4( +++ L 6L Ct
$F
s *CACC$C>Cc B \ W4 P p\ W4 # )u=
“Pull”-based encoder.
:param input: An iterable of Unicode strings.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
:returns: An iterable of byte strings.
)IncrementalEncoderr _iter_encode_generator)r5 r r7 r s &&& r iter_encoderR s
1
8
8F !% 00r
c # | " V F pV! V4 pV' g K Vx K V! R RR7 pV' d Vx R# R# 5i) TrJ N )r5 r rM rN s && r rQ rQ s= 6L Bd
#F
s <<<c 8 a ] tR tRt o RtRR ltRR ltRtV tR# ) rC i u3
“Push”-based decoder.
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does note have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
c \ \ V4 V n W n R V n RV n RV n R# )r
N)r _fallback_encoding_errors_buffer_decoderr )r$ r6 r7 s &&&r r% IncrementalDecoder.__init__ s, "/0A"B
r
c X V P pVe V! W4 # V P V, p\ V4 w rAVf- \ V4 ^8 d V' g Wn R# V P pVP
P
V P 4 P pW0n W@n V! W4 # )zDecode one chunk of the input.
:param input: A byte string.
:param final:
Indicate that no more input is available.
Must be :obj:`True` if this is the last call.
:returns: An Unicode string.
rT )
r[ rZ r4 lenrX r incrementaldecoderrY r r )r$ r5 rK rF r s &&& r r IncrementalDecoder.decode% s -- 5( ( u$%e, 5zA~e $ 22%%88 FMM
u$$r
)rZ r[ rY rX r Nreplace)F) r* r+ r, r- r. r% r r/ r0 r1 s @r rC rC s
% %r
rC c 2 a ] tR tRt o Rt]R3R ltRtV tR# )rP iA u
“Push”-based encoder.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`~exceptions.LookupError` for an unknown encoding label.
.. method:: encode(input, final=False)
:param input: An Unicode string.
:param final:
Indicate that no more input is available.
Must be :obj:`True` if this is the last call.
:returns: A byte string.
r@ c p \ V4 pVP P V4 P V n R # r# )r r incrementalencoderr )r$ r r7 s &&&r r% IncrementalEncoder.__init__R s* * ))< r r4 r rH rD rR rQ rC rP rU r
r rj s
Ln7 84<$+v +0 g * * B, G 8 @ !%X
1 0% 0%fL Lr