Force json module use to decode requests text response

When debugging tests execution of swh-graph, I stumbled across that error:

graph_client = <RemoteGraphClient url=http://127.0.0.1:54177/graph/>

    def test_stats(graph_client):
>       stats = graph_client.stats()

swh/graph/tests/test_api_client.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
swh/graph/client.py:36: in stats
    return self.get('stats')
../swh-core/swh/core/api/__init__.py:220: in get
    return self._decode_response(response)
../swh-core/swh/core/api/__init__.py:246: in _decode_response
    return decode_response(response)
../swh-core/swh/core/api/serializers.py:31: in decode_response
    r = response.json(cls=SWHJSONDecoder)
/usr/lib/python3/dist-packages/requests/models.py:889: in json
    self.content.decode(encoding), **kwargs
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

s = '{"counts":{"nodes":21,"edges":23},"ratios":{"compression":1.578,"bits_per_node":9.524,"bits_per_edge":8.696,"avg_loca...":3.696},"indegree":{"min":0,"max":3,"avg":1.0952380952380953},"outdegree":{"min":0,"max":3,"avg":1.0952380952380953}}', encoding = None
cls = <class 'swh.core.api.serializers.SWHJSONDecoder'>, object_hook = None, parse_float = None, parse_int = None, parse_constant = None, object_pairs_hook = None, use_decimal = False, kw = {}

    def loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,
            parse_int=None, parse_constant=None, object_pairs_hook=None,
            use_decimal=False, **kw):
        """Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON
        document) to a Python object.
    
        *encoding* determines the encoding used to interpret any
        :class:`str` objects decoded by this instance (``'utf-8'`` by
        default).  It has no effect when decoding :class:`unicode` objects.
    
        Note that currently only encodings that are a superset of ASCII work,
        strings of other encodings should be passed in as :class:`unicode`.
    
        *object_hook*, if specified, will be called with the result of every
        JSON object decoded and its return value will be used in place of the
        given :class:`dict`.  This can be used to provide custom
        deserializations (e.g. to support JSON-RPC class hinting).
    
        *object_pairs_hook* is an optional function that will be called with
        the result of any object literal decode with an ordered list of pairs.
        The return value of *object_pairs_hook* will be used instead of the
        :class:`dict`.  This feature can be used to implement custom decoders
        that rely on the order that the key and value pairs are decoded (for
        example, :func:`collections.OrderedDict` will remember the order of
        insertion). If *object_hook* is also defined, the *object_pairs_hook*
        takes priority.
    
        *parse_float*, if specified, will be called with the string of every
        JSON float to be decoded.  By default, this is equivalent to
        ``float(num_str)``. This can be used to use another datatype or parser
        for JSON floats (e.g. :class:`decimal.Decimal`).
    
        *parse_int*, if specified, will be called with the string of every
        JSON int to be decoded.  By default, this is equivalent to
        ``int(num_str)``.  This can be used to use another datatype or parser
        for JSON integers (e.g. :class:`float`).
    
        *parse_constant*, if specified, will be called with one of the
        following strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.  This
        can be used to raise an exception if invalid JSON numbers are
        encountered.
    
        If *use_decimal* is true (default: ``False``) then it implies
        parse_float=decimal.Decimal for parity with ``dump``.
    
        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
        kwarg. NOTE: You should use *object_hook* or *object_pairs_hook* instead
        of subclassing whenever possible.
    
        """
        if (cls is None and encoding is None and object_hook is None and
                parse_int is None and parse_float is None and
                parse_constant is None and object_pairs_hook is None
                and not use_decimal and not kw):
            return _default_decoder.decode(s)
        if cls is None:
            cls = JSONDecoder
        if object_hook is not None:
            kw['object_hook'] = object_hook
        if object_pairs_hook is not None:
            kw['object_pairs_hook'] = object_pairs_hook
        if parse_float is not None:
            kw['parse_float'] = parse_float
        if parse_int is not None:
            kw['parse_int'] = parse_int
        if parse_constant is not None:
            kw['parse_constant'] = parse_constant
        if use_decimal:
            if parse_float is not None:
                raise TypeError("use_decimal=True implies parse_float=Decimal")
            kw['parse_float'] = Decimal
>       return cls(encoding=encoding, **kw).decode(s)
E       TypeError: __init__() got an unexpected keyword argument 'encoding'

/usr/lib/python3/dist-packages/simplejson/__init__.py:535: TypeError

Indeed, when the simplejson module is present in the Phython environment (this is a pgadmin4 dependency for instance), requests will use it to decode json but a call to response.json(cls=...) will raise an exception as the arguments accepted by JSONDecoder are inconsistent between the standard library json and simplejson.

See https://github.com/psf/requests/issues/4842 for more details.

So ensure to use standard json module for decoding response texts and thus avoid possible errors when the simplejson module is present in the Python environment.

---

Migrated from D2177 (view on Phabricator)