U +if'@sNdZddlmZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZddlmZmZdd lmZmZmZmZerdd lmZmZmZmZmZd Z d d ddddZ!dde dfdddddddddZ"de fdddddddZ#d dddd!d"d#Z$de dfd$ddddd%d&d'Z%de dfdddddd%d(d)Z&dS)*z" data hash pandas / numpy objects ) annotationsN) TYPE_CHECKINGHashableIterableIteratorcast)lib)hash_object_array) ArrayLike)is_categorical_dtype is_list_like) ABCDataFrameABCIndex ABCMultiIndex ABCSeries) Categorical DataFrameIndex MultiIndexSeriesZ0123456789123456zIterator[np.ndarray]intz np.ndarray)arrays num_itemsreturncCsz t|}Wn$tk r0tjgtjdYSXt|g|}td}t|td}t|D]6\}}||}||N}||9}|td||7}qf|d|kst d|td7}|S)z Parameters ---------- arrays : Iterator[np.ndarray] num_items : int Returns ------- np.ndarray[uint64] Should be the same as CPython's tupleobject.c dtypeiCBixV4iXBzFed in wrong num_itemsi|) next StopIterationnparrayuint64 itertoolschainZ zeros_like enumerateAssertionError)rrfirstZmultoutiaZ inverse_ir*K/home/mars/bis/venv/lib/python3.8/site-packages/pandas/core/util/hashing.pycombine_hash_arrays.s   r,Tutf8zIndex | DataFrame | Seriesboolstrz str | Noner)objindexencodinghash_key categorizercsddlm}dkrtttr8|tdddSttrptjj ddd}||ddd}n tt rtjj ddd}|rȇfd d d D}t |g|} t | d }||jddd}nttrjfd d D} tj} |rLfdd d D} | d7} t | | } dd | D} t | | }||jddd}ntdt|S)a~ Return a data hash of the Index/Series/DataFrame. Parameters ---------- obj : Index, Series, or DataFrame index : bool, default True Include the index in the hash (if Series/DataFrame). encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- Series of uint64, same length as the object r)rNr!F)rcopyr5)r1rr5c3s$|]}tjddjVqdSF)r1r2r3r4Nhash_pandas_objectr1_values.0_r4r2r3r0r*r+ |sz%hash_pandas_object..Nc3s"|]\}}t|jVqdSr@) hash_arrayr:)r<r=Zseries)r4r2r3r*r+r?sc3s$|]}tjddjVqdSr7r8r;r>r*r+r?srcss|] }|VqdSr@r*)r<xr*r*r+r?szUnexpected type for hashing )pandasr_default_hash_key isinstancer hash_tuplesrrBr:astyperr"r#r,r1r itemslencolumns TypeErrortype)r0r1r2r3r4rhZserZ index_iterrhashesrZindex_hash_generator_hashesr*r>r+r9NsN           r9z+MultiIndex | Iterable[tuple[Hashable, ...]])valsr2r3rcszt|stdddlmm}t|ts6||n|fddtj D}fdd|D}t |t |}|S)a Hash an MultiIndex / listlike-of-tuples efficiently. Parameters ---------- vals : MultiIndex or listlike-of-tuples encoding : str, default 'utf8' hash_key : str, default _default_hash_key Returns ------- ndarray[np.uint64] of hashed values z'must be convertible to a list-of-tuplesr)rrcs(g|] }j|j|dddqS)FTZorderedZfastpath)codesZlevels)r<level)rmir*r+ szhash_tuples..c3s|]}t|dVqdS)r2r3N)_hash_categorical)r<catrWr*r+r?szhash_tuples..) r rLrDrrrFr from_tuplesrangeZnlevelsr,rJ)rQr2r3rZcat_valsrOrNr*)rr2r3rUr+rGs    rGr)rYr2r3rcCsdt|jj}t|||dd}|}t|r<||j}ntj t|dd}| r`t j ||<|S)a Hash a Categorical by hashing its categories, and then mapping the codes to the hashes Parameters ---------- cat : Categorical encoding : str hash_key : str Returns ------- ndarray[np.uint64] of hashed values, same size as len(c) F)r4r!r) rZasarray categoriesr:rBZisnarJZtakerSZzerosanyrZu8max)rYr2r3valueshashedmaskresultr*r*r+rXs  rXr )rQr2r3r4rcCs\t|dstd|j}t|r6td|}t|||St|tjsN| \}}t ||||S)aK Given a 1d array, return an array of deterministic integers. Parameters ---------- vals : ndarray or ExtensionArray encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- ndarray[np.uint64, ndim=1] Hashed values, same length as the vals. rzmust pass a ndarray-liker) hasattrrLrr rrXrFrZndarrayZ_values_for_factorize _hash_ndarray)rQr2r3r4rr=r*r*r+rBs     rBc Csv|j}t|tjr4tt|dtt|St|trJ| d}nt |j tj tj frt|dj ddd}nt |j tjr|jdkr|d|jj d}n|rdd lm}m}m}||dd \}} |||| dd d } t| ||Szt|||}Wn.tk r0t| t t||}YnX||d ?N}|td9}||d?N}|td9}||d?N}|S)z! See hash_array.__doc__. u8i8Fr6ur)rr factorize)sortTrRle9z lb&&& )rrZ issubdtypeZ complex128rBrealimagrFr.rH issubclassrMZ datetime64Z timedelta64viewnumberitemsizerDrrriZ _with_inferrXr rLr/objectr!) rQr2r3r4rrrrirSr\rYr*r*r+rc(sB         rc)'__doc__ __future__rr"typingrrrrrnumpyrZ pandas._libsrZpandas._libs.hashingr Zpandas._typingr Zpandas.core.dtypes.commonr r Zpandas.core.dtypes.genericr rrrrDrrrrrrEr,r9rGrXrBrcr*r*r*r+s<     "^.(,