{"id":11543,"date":"2024-08-15T18:45:56","date_gmt":"2024-08-15T17:45:56","guid":{"rendered":"https:\/\/www.blopig.com\/blog\/?p=11543"},"modified":"2024-08-18T11:52:09","modified_gmt":"2024-08-18T10:52:09","slug":"sort-and-slice-tutorial-an-alternative-to-extended-connectivity-fingerprints","status":"publish","type":"post","link":"https:\/\/www.blopig.com\/blog\/2024\/08\/sort-and-slice-tutorial-an-alternative-to-extended-connectivity-fingerprints\/","title":{"rendered":"Sort and Slice Tutorial – An alternative to extended connectivity fingerprints"},"content":{"rendered":"\n
\n\n
\n
\n
\n<\/div>\n
\n<\/div>
\n

Background\u00b6<\/a><\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n<\/div>\n
\n<\/div>
\n

Sort and Slice<\/a> (SNS) was developed by a former OPIGlet, Markus, as a method for improving Extended Connectivity Fingerprints (ECFPs)<\/a> by overcoming bit collisions. ECFPs are a form of topological fingerprint which denote the absence and presence of circular substructures in a molecule. The steps for deriving an ECFP from a molecule are as follows:<\/p>\n

    \n

  1. Identifier assignment:<\/p>\n

    Each atom in the molecule is assigned an initial numerical identifier; this is typically generated by hashing a tuple of atomic properties called Daylight atomic invariants<\/a> into a 32-bit integer. These properties are:<\/p>\n

      \n
    1. Number of non-hydrogen neighbours.<\/li>\n
    2. Valence – number of neighbouring hydrogens.<\/li>\n
    3. Atomic number.<\/li>\n
    4. Atomic mass.<\/li>\n
    5. Atomic charge.<\/li>\n
    6. Number of hydrogen neighbours.<\/li>\n
    7. Ring membership.*<\/li>\n<\/ol>\n

      *Ring membership is an additional property that is often used but is not one of the original Daylight atomic invariants.<\/p>\n<\/li>\n\n\n\n\n\n\n\n

    8. Iterative updating:<\/p>\n

      Each substructure identifier is updated by hashing the central identifier with all the identifiers of its neighbours, i.e. performing a 1-hop aggregation. This processes is repeated upto a predetermined maximum radius.<\/p>\n<\/li>\n

    9. Duplicate removal:<\/p>\n

      Duplicate substructures are identified and removed.<\/p>\n<\/li>\n

    10. Folding into a smaller bit vector:<\/p>\n

      The 32-bit integers are used to produce a sparse vector, with ‘on’ bits for each substructure identifier in the molecule. This sparse bit vector is folded into smaller fixed length bit vector, typically to 1024 or 2048 bits. This final folding step often produces ‘bit collisions’, with the same bit being activated by different substructures, adding noise to the representation.<\/p>\n<\/li>\n<\/ol>\n

      For background more on ECFPs, I recommend this paper by Rogers and Hahn (2010)<\/a> which introduced ECFPs, Leo’s blopig post on topological fingerprints<\/a>, Markus’ blopig post on turning SMILES into ECFPs with RDKit<\/a>, and Greg Landrum’s blog post on fingerprint generators in RDKit<\/a>.<\/p>\n

      SNS is a form of dataset dependent feature selection applied to ECFPs. Instead of folding the final sparse vector, SNS retains only the most frequent substructures in a previously observed dataset. Let’s see how we might do this in Python.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>

      \n
      \n
      \n<\/div>\n
      \n
      In\u00a0[1]:<\/div>\n
      \n
      \n
      <\/span># imports<\/span>\nimport<\/span> pandas<\/span> as<\/span> pd<\/span>\nfrom<\/span> rdkit.Chem.rdFingerprintGenerator<\/span> import<\/span> AdditionalOutput<\/span>,<\/span> GetMorganGenerator<\/span>\nimport<\/span> numpy<\/span> as<\/span> np<\/span>\nfrom<\/span> sklearn.model_selection<\/span> import<\/span> train_test_split<\/span>\nfrom<\/span> rdkit<\/span> import<\/span> Chem<\/span>\nfrom<\/span> tqdm<\/span> import<\/span> tqdm<\/span> # progress bar<\/span>\nimport<\/span> jdc<\/span> # allows dynamic class definition across cells<\/span>\nimport<\/span> warnings<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n<\/div>\n
      \n<\/div>
      \n
      Data\u00b6<\/a><\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n<\/div>\n
      \n<\/div>
      \n
      First of all we need a dataset to test our code on. QM9<\/a> is a commonly used dataset of small molecules with calculated quantum properties, which can be easily downloaded from MoleculeNet<\/a> using pandas:<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
      \n
      \n
      \n<\/div>\n
      \n
      In\u00a0[2]:<\/div>\n
      \n
      \n
      <\/span>qm9<\/span> =<\/span> 'https:\/\/deepchemdata.s3-us-west-1.amazonaws.com\/datasets\/qm9.csv'<\/span>\nqm9<\/span> =<\/span> pd<\/span>.<\/span>read_csv<\/span>(<\/span>qm9<\/span>)<\/span>\nqm9<\/span>.<\/span>head<\/span>()<\/span>.<\/span>iloc<\/span>[:,:<\/span>2<\/span>]<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n<\/div>\n
      \n
      \n
      Out[2]:<\/div>\n
      \n
      \n\n\n\n\n\n\n\n\n\n
      <\/th>\nmol_id<\/th>\nsmiles<\/th>\n<\/tr>\n<\/thead>\n
      0<\/th>\ngdb_1<\/td>\nC<\/td>\n<\/tr>\n
      1<\/th>\ngdb_2<\/td>\nN<\/td>\n<\/tr>\n
      2<\/th>\ngdb_3<\/td>\nO<\/td>\n<\/tr>\n
      3<\/th>\ngdb_4<\/td>\nC#C<\/td>\n<\/tr>\n
      4<\/th>\ngdb_5<\/td>\nC#N<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n<\/div>\n
      \n<\/div>
      \n
      Let’s perform a train\/test split and convert the SMILES strings into RDKit molecule objects. This will give us train data to generate an SNS featurizer from and test data to run the SNS featurizer over.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
      \n
      \n
      \n<\/div>\n
      \n
      In\u00a0[3]:<\/div>\n
      \n
      \n
      <\/span>train<\/span>,<\/span> test<\/span> =<\/span> train_test_split<\/span>(<\/span>qm9<\/span>,<\/span> test_size<\/span>=<\/span>0.4<\/span>,<\/span> random_state<\/span>=<\/span>42<\/span>)<\/span>\n\ntrain_smiles<\/span> =<\/span> train<\/span>[<\/span>'smiles'<\/span>]<\/span>.<\/span>to_list<\/span>()<\/span>\ntest_smiles<\/span> =<\/span> test<\/span>[<\/span>'smiles'<\/span>]<\/span>.<\/span>to_list<\/span>()<\/span>\n\ntrain_mols<\/span> =<\/span> [<\/span>Chem<\/span>.<\/span>MolFromSmiles<\/span>(<\/span>smi<\/span>)<\/span> for<\/span> smi<\/span> in<\/span> train_smiles<\/span>]<\/span>\ntest_mols<\/span> =<\/span> [<\/span>Chem<\/span>.<\/span>MolFromSmiles<\/span>(<\/span>smi<\/span>)<\/span> for<\/span> smi<\/span> in<\/span> test_smiles<\/span>]<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n<\/div>\n
      \n<\/div>
      \n
      Coding Sort and Slice\u00b6<\/a><\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      \n<\/div>\n
      \n<\/div>
      \n
      Now we have our data, how can we build a SortAndSlice<\/code> class? There are several components our class needs to possess:<\/p>\n
        \n
      1. An RDKit Morgan Fingerprint generator for performing substructure identifier generation.<\/li>\n
      2. An RDKit AdditionalOutput object for obtaining substructure identifiers from the Morgan generator.<\/li>\n
      3. A method for obtaining all the substructure identifiers in the training dataset.<\/li>\n
      4. A dictionary for holding all the identifiers in the training dataset.<\/li>\n
      5. An encoding dictionary of a fixed length for mapping substructures to bits.<\/li>\n
      6. A method for converting new molecules to SNS bit vectors.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        \n
        \n
        \n<\/div>\n
        \n<\/div>
        \n
        Class Constructor\u00b6<\/a><\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        \n
        \n
        \n<\/div>\n
        \n<\/div>
        \n
        Below is the constructor method of our SortAndSlice<\/code> class. This method takes a list of training data molecules, a Morgan Generator, the desired bit length, and a verbosity flag as input arguments. The constructor defines a fingerprint generator and AdditionalOutput instance as attributes of our class, before computing the identifiers in the dataset and the encoder.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
        \n
        \n
        \n<\/div>\n
        \n
        In\u00a0[4]:<\/div>\n
        \n
        \n
        <\/span>class<\/span> SortAndSlice<\/span>:<\/span>\n\n    encoder<\/span> =<\/span> {}<\/span>\n\n    def<\/span> __init__<\/span>(<\/span>\n        self<\/span>,<\/span>\n        data<\/span>:<\/span> list<\/span>[<\/span>Chem<\/span>.<\/span>Mol<\/span>],<\/span>\n        generator<\/span>:<\/span> GetMorganGenerator<\/span> =<\/span> GetMorganGenerator<\/span>(<\/span>radius<\/span>=<\/span>2<\/span>),<\/span>\n        bit_length<\/span>:<\/span> int<\/span> =<\/span> 2048<\/span>,<\/span>\n        verbose<\/span>:<\/span> bool<\/span> =<\/span> False<\/span>,<\/span>\n    ):<\/span>\n        self<\/span>.<\/span>generator<\/span> =<\/span> generator<\/span>\n        self<\/span>.<\/span>ao<\/span>:<\/span> AdditionalOutput<\/span> =<\/span> AdditionalOutput<\/span>()<\/span>\n        self<\/span>.<\/span>ao<\/span>.<\/span>AllocateBitInfoMap<\/span>()<\/span>\n        self<\/span>.<\/span>verbose<\/span> =<\/span> verbose<\/span>\n\n        self<\/span>.<\/span>identifiers<\/span> =<\/span> self<\/span>.<\/span>get_identifiers<\/span>(<\/span>data<\/span>)<\/span>\n        self<\/span>.<\/span>set_encoder<\/span>(<\/span>bit_length<\/span>)<\/span>\n\n    \n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        \n
        \n
        \n<\/div>\n
        \n<\/div>
        \n
        Sort\u00b6<\/a><\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        \n
        \n
        \n<\/div>\n
        \n<\/div>
        \n
        With our constructor written, the next requirement is a method which calculates the frequency of substructure identifiers across our dataset and sorts the identifiers by frequency. The method below, get_identifiers<\/code>, does the following:<\/p>\n
          \n
        1. Creates an empty dictionary to tally substructure identifiers.<\/li>\n
        2. Iterates over each molecule in the input data. For each iteration:
            \n
          1. Calculates the sparse Morgan fingerprint of a molecule.<\/li>\n
          2. Obtains all the substructure integer identifiers from the additional output object.<\/li>\n
          3. For each integer, adds 1 to the total tally.<\/li>\n<\/ol>\n<\/li>\n
          4. Sorts the identifiers by frequency.<\/li>\n
          5. Returns the identifiers as a dictionary, with identifiers as keys and tallies as values.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
            \n
            \n
            \n<\/div>\n
            \n
            In\u00a0[5]:<\/div>\n
            \n
            \n
            <\/span>%%<\/span>add_to<\/span> SortAndSlice\ndef get_identifiers(self, data: list[Chem.Mol]) -> dict[int, int]:\n    identifiers = {}\n    pbar = tqdm(total=len(data), desc='Collecting identifiers', disable=not self.verbose)\n    for mol in data:\n        self.generator.GetSparseFingerprint(mol, additionalOutput=self.ao)\n        bitmap = self.ao.GetBitInfoMap()\n        for identifier in bitmap:\n            count = identifiers.get(identifier, 0)\n            identifiers[identifier] = count + 1\n        pbar.update(1)\n    pbar.close()\n    identifiers = dict(sorted(identifiers.items(), key=lambda x: x[1], reverse=True))\n    return identifiers\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
            \n
            \n
            \n<\/div>\n
            \n<\/div>
            \n
            Slice\u00b6<\/a><\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
            \n
            \n
            \n<\/div>\n
            \n<\/div>
            \n
            Having computed the identifiers in our dataset and sorted them by frequency, the slicing operation comes next. The method set_encoder<\/code> takes an integer, bit_length<\/code>, as an argument. The SortAndSlice<\/code> encoder is then set by taking only the top bit_length<\/code> substructures by frequency and labelling them with a rank, i.e. the identifier dictionary is sliced and enumerated.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
            \n
            \n
            \n<\/div>\n
            \n
            In\u00a0[6]:<\/div>\n
            \n
            \n
            <\/span>%%<\/span>add_to<\/span> SortAndSlice\ndef set_encoder(self, bit_length: int):\n    if self.verbose:\n        print(f'Setting bit length of encoder to a max of {bit_length}.')\n    encoder = {}\n    for i, k in enumerate(self.identifiers.keys()):\n        if i >= bit_length:\n            break\n        encoder[k] = i\n\n    self.encoder = encoder\n    if len(encoder) < bit_length:\n       warnings.warn(f'Encoder is only {len(encoder)} bits long.')\n\n    if self.verbose:\n        print(f'Encoder set to {len(encoder)} bits.')\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
            \n
            \n
            \n<\/div>\n
            \n<\/div>
            \n
            Featurization\u00b6<\/a><\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
            \n
            \n
            \n<\/div>\n
            \n<\/div>
            \n
            With an encoder dictionary now set, new molecules can be encoded based on the molecules that have already been seen. The encode<\/code> method takes an rdkit molecule as an argument and performs the following:<\/p>\n
              \n
            1. Calculates the sparse morgan fingerprint of the molecule.<\/li>\n
            2. Obtains the 32-bit substructure identifiers in the molecule.<\/li>\n
            3. Generates an output vector of zeros that’s the length of the encoder.<\/li>\n
            4. Iterates over the identifiers in the molecule. For each identifier:
                \n
              1. Checks if the identifier is in the encoder.<\/li>\n
              2. If the identifier is encoded, the rank of the identifier is used to turn a bit in the output vector to 1.<\/li>\n<\/ol>\n<\/li>\n
              3. Returns the SNS feature vector for the molecule.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
                \n
                \n
                \n<\/div>\n
                \n
                In\u00a0[7]:<\/div>\n
                \n
                \n
                <\/span>%%<\/span>add_to<\/span> SortAndSlice\ndef encode(self, mol: Chem.Mol) -> np.ndarray:\n    self.generator.GetSparseFingerprint(mol, additionalOutput=self.ao)\n    bitmap = self.ao.GetBitInfoMap()\n    out = np.zeros(len(self.encoder))\n    for identifier in bitmap:\n        if identifier in self.encoder:\n            out[self.encoder[identifier]] = 1\n    return out\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                Usage\u00b6<\/a><\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                Let’s see how this SortAndSlice<\/code> class can be used. Below, a Morgan Fingerprint generator is defined. Next, this generator and the QM9 training molecules are used as arguments for an instance of the SortAndSlice class:<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
                \n
                \n
                \n<\/div>\n
                \n
                In\u00a0[8]:<\/div>\n
                \n
                \n
                <\/span>gen<\/span> =<\/span> GetMorganGenerator<\/span>(<\/span>radius<\/span>=<\/span>2<\/span>,<\/span> includeChirality<\/span>=<\/span>True<\/span>)<\/span>\nsns<\/span> =<\/span> SortAndSlice<\/span>(<\/span>train_mols<\/span>,<\/span> generator<\/span>=<\/span>gen<\/span>,<\/span> bit_length<\/span>=<\/span>2048<\/span>,<\/span> verbose<\/span>=<\/span>True<\/span>)<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n<\/div>\n
                \n
                \n
                <\/div>\n
                \n
                Collecting identifiers: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 80331\/80331 [00:02<00:00, 31337.66it\/s]<\/pre>\n<\/div>\n<\/div>\n
                \n
                <\/div>\n
                \n
                Setting bit length of encoder to a max of 2048.\nEncoder set to 2048 bits.\n<\/pre>\n<\/div>\n<\/div>\n
                \n
                <\/div>\n
                \n
                \n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                To use this SortAndSlice<\/code> instance to featurize a set of molecules, we can do the following:<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
                \n
                \n
                \n<\/div>\n
                \n
                In\u00a0[9]:<\/div>\n
                \n
                \n
                <\/span>test_sort_and_slice<\/span> =<\/span> [<\/span>sns<\/span>.<\/span>encode<\/span>(<\/span>mol<\/span>)<\/span> for<\/span> mol<\/span> in<\/span> test_mols<\/span>]<\/span>\nprint<\/span>(<\/span>\n    f<\/span>'Number of feature vectors: <\/span>{<\/span>len<\/span>(<\/span>test_sort_and_slice<\/span>)<\/span>}<\/span>,<\/span>\\n\\<\/span>\n    Length of the 0th feature vector: <\/span>{<\/span>test_sort_and_slice<\/span>[<\/span>0<\/span>]<\/span>.<\/span>shape<\/span>[<\/span>0<\/span>]<\/span>}<\/span>,<\/span>\\n\\<\/span>\n    On bits in the 0th feature vector: <\/span>{<\/span>np<\/span>.<\/span>where<\/span>(<\/span>test_sort_and_slice<\/span>[<\/span>0<\/span>])[<\/span>0<\/span>]<\/span>}<\/span>'<\/span>\n)<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n<\/div>\n
                \n
                \n
                <\/div>\n
                \n
                Number of feature vectors: 53554,\n    Length of the 0th feature vector: 2048,\n    On bits in the 0th feature vector: [   0    1    4    5   10   20   21   22   35   53   69  100  629  901\n 1242]\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                And there you have it! A set of topological fingerprints without bit collisions!<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                For clarity, here’s how the SortAndSlice class looks without cell breaks:<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>
                \n
                \n
                \n<\/div>\n
                \n
                In\u00a0[10]:<\/div>\n
                \n
                \n
                <\/span>class<\/span> SortAndSlice<\/span>:<\/span>\n    <\/span>\"\"\"<\/span>\n    Class to sort and slice the output of a Morgan fingerprint generator.<\/span>\n<\/span>\n    Args:<\/span>\n        data (list[Chem.Mol]): List of RDKit molecules.<\/span>\n        generator (GetMorganGenerator): RDKit Morgan fingerprint generator.<\/span>\n        bit_length (int): Length of the output vector.<\/span>\n        verbose (bool): Whether to print progress.<\/span>\n\n    Attributes:<\/span>\n        generator (GetMorganGenerator): RDKit Morgan fingerprint generator.<\/span>\n        ao (AdditionalOutput): RDKit fingerprint generator output.<\/span>\n        verbose (bool): Whether to print progress.<\/span>\n        identifiers (dict[int, int]): Dictionary of identifiers and counts.<\/span>\n        encoder (dict[int, int]): Dictionary of identifiers and indices.<\/span>\n\n    Methods:<\/span>\n        get_identifiers: Collects identifiers from data.<\/span>\n        set_encoder: Sets the encoder to a specific length.<\/span>\n        encode: Encodes a molecule into a binary vector.<\/span>\n    \"\"\"<\/span>\n    encoder<\/span> =<\/span> {}<\/span>\n\n    def<\/span> __init__<\/span>(<\/span>\n        self<\/span>,<\/span>\n        data<\/span>:<\/span> list<\/span>[<\/span>Chem<\/span>.<\/span>Mol<\/span>],<\/span>\n        generator<\/span>:<\/span> GetMorganGenerator<\/span> =<\/span> GetMorganGenerator<\/span>(<\/span>radius<\/span>=<\/span>2<\/span>),<\/span>\n        bit_length<\/span>:<\/span> int<\/span> =<\/span> 2048<\/span>,<\/span>\n        verbose<\/span>:<\/span> bool<\/span> =<\/span> False<\/span>,<\/span>\n    ):<\/span>\n        self<\/span>.<\/span>generator<\/span> =<\/span> generator<\/span>\n        self<\/span>.<\/span>ao<\/span>:<\/span> AdditionalOutput<\/span> =<\/span> AdditionalOutput<\/span>()<\/span>\n        self<\/span>.<\/span>ao<\/span>.<\/span>AllocateBitInfoMap<\/span>()<\/span>\n        self<\/span>.<\/span>verbose<\/span> =<\/span> verbose<\/span>\n\n        self<\/span>.<\/span>identifiers<\/span> =<\/span> self<\/span>.<\/span>get_identifiers<\/span>(<\/span>data<\/span>)<\/span>\n        self<\/span>.<\/span>set_encoder<\/span>(<\/span>bit_length<\/span>)<\/span>\n\n    def<\/span> get_identifiers<\/span>(<\/span>self<\/span>,<\/span> data<\/span>:<\/span> list<\/span>[<\/span>Chem<\/span>.<\/span>Mol<\/span>])<\/span> -><\/span> dict<\/span>[<\/span>str<\/span>,<\/span> int<\/span>]:<\/span>\n        <\/span>\"\"\"<\/span>\n        Collects and sorts identifiers from molecule data.<\/span>\n        <\/span>\n        Args:<\/span>\n            data (list[Chem.Mol]): List of RDKit molecules.<\/span>\n            <\/span>\n        Returns:<\/span>\n            dict[int, int]: Dictionary of identifiers and counts.<\/span>\n        \"\"\"<\/span>\n        identifiers<\/span> =<\/span> {}<\/span>\n        pbar<\/span> =<\/span> tqdm<\/span>(<\/span>total<\/span>=<\/span>len<\/span>(<\/span>data<\/span>),<\/span> desc<\/span>=<\/span>'Collecting identifiers'<\/span>,<\/span> disable<\/span>=<\/span>not<\/span> self<\/span>.<\/span>verbose<\/span>)<\/span>\n        for<\/span> mol<\/span> in<\/span> data<\/span>:<\/span>\n            self<\/span>.<\/span>generator<\/span>.<\/span>GetSparseFingerprint<\/span>(<\/span>mol<\/span>,<\/span> additionalOutput<\/span>=<\/span>self<\/span>.<\/span>ao<\/span>)<\/span>\n            bitmap<\/span> =<\/span> self<\/span>.<\/span>ao<\/span>.<\/span>GetBitInfoMap<\/span>()<\/span>\n            for<\/span> identifier<\/span> in<\/span> bitmap<\/span>:<\/span>\n                count<\/span> =<\/span> identifiers<\/span>.<\/span>get<\/span>(<\/span>identifier<\/span>,<\/span> 0<\/span>)<\/span>\n                identifiers<\/span>[<\/span>identifier<\/span>]<\/span> =<\/span> count<\/span> +<\/span> 1<\/span>\n            pbar<\/span>.<\/span>update<\/span>(<\/span>1<\/span>)<\/span>\n        pbar<\/span>.<\/span>close<\/span>()<\/span>\n        identifiers<\/span> =<\/span> dict<\/span>(<\/span>sorted<\/span>(<\/span>identifiers<\/span>.<\/span>items<\/span>(),<\/span> key<\/span>=<\/span>lambda<\/span> x<\/span>:<\/span> x<\/span>[<\/span>1<\/span>],<\/span> reverse<\/span>=<\/span>True<\/span>))<\/span>\n        return<\/span> identifiers<\/span>\n    \n    def<\/span> set_encoder<\/span>(<\/span>self<\/span>,<\/span> bit_length<\/span>:<\/span> int<\/span>):<\/span>\n        <\/span>\"\"\"<\/span>\n        Slices substructure identifiers to a specific length enumerates them.<\/span>\n        Sets the encoder attribute to the enumerated identifiers.<\/span>\n\n        Args:<\/span>\n            bit_length (int): Length of the output vector.<\/span>\n        \"\"\"<\/span>\n        if<\/span> self<\/span>.<\/span>verbose<\/span>:<\/span>\n            print<\/span>(<\/span>f<\/span>'Setting bit length of encoder to a max of <\/span>{<\/span>bit_length<\/span>}<\/span>.'<\/span>)<\/span>\n        encoder<\/span> =<\/span> {}<\/span>\n        for<\/span> i<\/span>,<\/span> k<\/span> in<\/span> enumerate<\/span>(<\/span>self<\/span>.<\/span>identifiers<\/span>.<\/span>keys<\/span>()):<\/span>\n            if<\/span> i<\/span> >=<\/span> bit_length<\/span>:<\/span>\n                break<\/span>\n            encoder<\/span>[<\/span>k<\/span>]<\/span> =<\/span> i<\/span>\n\n        self<\/span>.<\/span>encoder<\/span> =<\/span> encoder<\/span>\n        if<\/span> len<\/span>(<\/span>encoder<\/span>)<\/span> <<\/span> bit_length<\/span>:<\/span>\n            warnings<\/span>.<\/span>warn<\/span>(<\/span>f<\/span>'Encoder is only <\/span>{<\/span>len<\/span>(<\/span>encoder<\/span>)<\/span>}<\/span> bits long.'<\/span>)<\/span>\n\n        if<\/span> self<\/span>.<\/span>verbose<\/span>:<\/span>\n            print<\/span>(<\/span>f<\/span>'Encoder set to <\/span>{<\/span>len<\/span>(<\/span>encoder<\/span>)<\/span>}<\/span> bits.'<\/span>)<\/span>\n\n    def<\/span> encode<\/span>(<\/span>self<\/span>,<\/span> mol<\/span>:<\/span> Chem<\/span>.<\/span>Mol<\/span>)<\/span> -><\/span> np<\/span>.<\/span>ndarray<\/span>:<\/span>\n        <\/span>\"\"\"<\/span>\n        Encodes a molecule into a binary sort and slice vector.<\/span>\n\n        Args:<\/span>\n            mol (Chem.Mol): RDKit molecule.<\/span>\n\n        Returns:<\/span>\n            np.ndarray: Binary vector indicating substructure presence.<\/span>\n        \"\"\"<\/span>\n        self<\/span>.<\/span>generator<\/span>.<\/span>GetSparseFingerprint<\/span>(<\/span>mol<\/span>,<\/span> additionalOutput<\/span>=<\/span>self<\/span>.<\/span>ao<\/span>)<\/span>\n        bitmap<\/span> =<\/span> self<\/span>.<\/span>ao<\/span>.<\/span>GetBitInfoMap<\/span>()<\/span>\n        out<\/span> =<\/span> np<\/span>.<\/span>zeros<\/span>(<\/span>len<\/span>(<\/span>self<\/span>.<\/span>encoder<\/span>))<\/span>\n        for<\/span> identifier<\/span> in<\/span> bitmap<\/span>:<\/span>\n            if<\/span> identifier<\/span> in<\/span> self<\/span>.<\/span>encoder<\/span>:<\/span>\n                out<\/span>[<\/span>self<\/span>.<\/span>encoder<\/span>[<\/span>identifier<\/span>]]<\/span> =<\/span> 1<\/span>\n        return<\/span> out<\/span>\n<\/pre><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
                \n
                \n
                \n<\/div>\n
                \n<\/div>
                \n
                References:\u00b6<\/a><\/h3>
                  \n
                1. Dablander, M. et al. (2024) Sort and Slice: A Simple and Superior Alternative to Hash-Based Folding for Extended-Connectivity Fingerprints. https:\/\/doi.org\/10.48550\/arXiv.2403.17954<\/a><\/li>\n
                2. Rogers, D. and Hahn, M. (2010) Extended-Connectivity Fingerprints. https:\/\/doi.org\/10.1021\/ci100050t<\/a><\/li>\n
                3. Weininger, D., Weininger, A., Weininger, J.L. (1989) SMILES. 2. Algorithm for generation of unique SMILES notation. https:\/\/doi.org\/10.1021\/ci00062a008<\/a><\/li>\n
                4. Klarner, L. (2022) Exploring Topological Fingerprints. https:\/\/www.blopig.com\/blog\/2022\/06\/exploring-topological-fingerprints-in-rdkit\/<\/a><\/li>\n
                5. Dablander, M. (2022) How to turn a SMILES string into an extended-connectivity fingerprint using RDKit. https:\/\/www.blopig.com\/blog\/2022\/11\/how-to-turn-a-smiles-string-into-an-extended-connectivity-fingerprint-using-rdkit\/<\/a><\/li>\n
                6. Landrum, G. (2023) FingerprintGenerator tutorial. https:\/\/greglandrum.github.io\/rdkit-blog\/posts\/2023-01-18-fingerprint-generator-tutorial.html<\/a><\/li>\n
                7. Ramakrishnan, R. et al (2014) Quantum chemistry structures and properties of 134 kilo molecules. https:\/\/doi.org\/10.1038\/sdata.2014.22<\/a><\/li>\n
                8. Wu, Z. et al. (2018) MoleculeNet: a benchmark for molecular machine learning. https:\/\/pubs.rsc.org\/en\/content\/articlelanding\/2018\/sc\/c7sc02664a<\/a><\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/main>\n
                  WordPress conversion from tutorial.ipynb by nb2wp<\/A> v0.3.1<\/p>\n\n\n\n
                  GitHub repo: https:\/\/github.com\/smkyrle\/sort-and-slice-tutorial<\/a><\/p>\n\n\n\n
                  <\/p>\n","protected":false},"excerpt":{"rendered":"
                  Background\u00b6 Sort and Slice (SNS) was developed by a former OPIGlet, Markus, as a method for improving Extended Connectivity Fingerprints (ECFPs) by overcoming bit collisions. ECFPs are a form of topological fingerprint which denote the absence and presence of circular substructures in a molecule. The steps for deriving an ECFP from a molecule are as […]<\/p>\n","protected":false},"author":118,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","wikipediapreview_detectlinks":true,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"ngg_post_thumbnail":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[187,227,201],"tags":[711,680,152,134,786],"ppma_author":[760],"class_list":["post-11543","post","type-post","status-publish","format-standard","hentry","category-cheminformatics","category-python-code","category-small-molecules","tag-ecfps","tag-extended-connectivity-fingerprints","tag-python","tag-small-molecules","tag-sort-and-slice"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"authors":[{"term_id":760,"user_id":118,"is_guest":0,"slug":"sam","display_name":"Sam Money-Kyrle","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/784870e2ed5304f12f11366dad56cbf1c0b9aa63bd80021ae235ba5f30536a12?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/11543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/users\/118"}],"replies":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/comments?post=11543"}],"version-history":[{"count":6,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/11543\/revisions"}],"predecessor-version":[{"id":12102,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/11543\/revisions\/12102"}],"wp:attachment":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/media?parent=11543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/categories?post=11543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/tags?post=11543"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=11543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}