.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/datasets_and_pipelines/gridsearch.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_datasets_and_pipelines_gridsearch.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_datasets_and_pipelines_gridsearch.py:


.. _grid_search:

Grid Search optimal Algorithm Parameter
=======================================

.. note:: These examples are basically copies from the same examples in tpcp, but using gait algorithms!
          These examples are less often updated than the official tpcp examples.
          Hence, it makes sense to cross-check the official examples.

.. GENERATED FROM PYTHON SOURCE LINES 12-16

.. code-block:: default


    import pandas as pd









.. GENERATED FROM PYTHON SOURCE LINES 17-36

To perform a GridSearch (or any other form of parameter optimization in Gaitmap), we first need to have a
**Dataset**, a **Pipeline** and a **score** function.

1. The Dataset
--------------
Datsets wrap multiple gait recordings into an easy to use interface that can be passed around between the higher
level gaitmap functions.
Learn more about this :ref:`here <custom_dataset>`.
If you are lucky, you do not need to create the dataset on your own, but someone has already created a gaitmap dataset
for the data you want to use.

Here we are going to create a simple "dummy" dataset, that just uses the left and the right-foot recording of the
example data to simulate a dataset of one participant with two recordings.
In addition to the raw IMU data, the dataset also exposes reference stride borders that we will use to judge the
performance of our algorithm.
(Note, usually, you wouldn't split the left and the right foot into separate recordings, as gaitmap can handle
multiple sensor recordings at once).

For our GridSearch, we need an instance of this dataset.

.. GENERATED FROM PYTHON SOURCE LINES 36-64

.. code-block:: default

    from tpcp import Dataset

    from gaitmap.example_data import get_healthy_example_imu_data, get_healthy_example_stride_borders
    from gaitmap.utils.datatype_helper import SingleSensorStrideList


    class MyDataset(Dataset):
        @property
        def sampling_rate_hz(self) -> float:
            return 204.8

        @property
        def data(self):
            self.assert_is_single(None, "data")
            return get_healthy_example_imu_data()[self.index.iloc[0]["foot"] + "_sensor"]

        @property
        def segmented_stride_list_(self) -> pd.DataFrame:
            self.assert_is_single(None, "data")
            return get_healthy_example_stride_borders()[self.index.iloc[0]["foot"] + "_sensor"].set_index("s_id")

        def create_index(self) -> pd.DataFrame:
            return pd.DataFrame({"participant": ["test", "test"], "foot": ["left", "right"]})


    dataset = MyDataset()
    dataset






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <h4 style="margin-bottom: 0.1em;">MyDataset [2 groups/rows]</h3>
    <div style="margin-top: 0em">
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table style="margin-left: 3em;">
      <thead>
        <tr style="text-align: right;">
          <th style="text-align: center;"></th>
          <th style="text-align: center;">participant</th>
          <th style="text-align: center;">foot</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th style="text-align: center;">0</th>
          <td style="text-align: center; padding-left: 2em; padding-right: 2em;">test</td>
          <td style="text-align: center; padding-left: 2em; padding-right: 2em;">left</td>
        </tr>
        <tr>
          <th style="text-align: center;">1</th>
          <td style="text-align: center; padding-left: 2em; padding-right: 2em;">test</td>
          <td style="text-align: center; padding-left: 2em; padding-right: 2em;">right</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 65-83

2. The Pipeline
---------------
The pipeline simply defines what algorithms we want to run on our data and defines, which parameters of the pipeline
you still want to be able to modify (e.g. to optimize in the GridSearch).

The pipeline usually needs 3 things:

1. It needs to be subclass of `SimplePipeline`.
2. It needs to have a `run` method that runs all the algorithmic steps and stores the results as class attributes.
   The `run` method should expect only a single data point (in our case a single recording of one sensor) as input.
3. A `init` that defines all parameters that should be adjustable. Note, that the names in the function signature of
   the `init` method, **must** match the corresponding attribute names (e.g. `max_cost` -> `self.max_cost`).

Here we simply transform the data into the correct coordinate system depending on the foot and apply `BarthDtw` to
identify the start and the end of all strides in the recording.
The parameter `max_cost` of this algorithm is exposed and will be optimized as part of the GridSearch.

For the final GridSearch, we need an instance of the pipeline object.

.. GENERATED FROM PYTHON SOURCE LINES 83-113

.. code-block:: default

    from tpcp import Pipeline

    from gaitmap.stride_segmentation import BarthDtw
    from gaitmap.utils.coordinate_conversion import convert_left_foot_to_fbf, convert_right_foot_to_fbf


    class MyPipeline(Pipeline):
        max_cost: float

        segmented_stride_list_: SingleSensorStrideList

        def __init__(self, max_cost: float = 3) -> None:
            self.max_cost = max_cost

        def run(self, datapoint: MyDataset):
            converter = {"left": convert_left_foot_to_fbf, "right": convert_right_foot_to_fbf}
            # `datapoint.groups[0]` gives us the identifier of the datapoint (e.g. `("test", "left")`).
            # And ``datapoint.groups[0][1]` is the foot.
            data = converter[datapoint.groups[0][1]](datapoint.data)

            dtw = BarthDtw(max_cost=self.max_cost)
            dtw.segment(data, datapoint.sampling_rate_hz)

            self.segmented_stride_list_ = dtw.stride_list_
            return self


    pipe = MyPipeline()









.. GENERATED FROM PYTHON SOURCE LINES 114-138

3. The scorer
-------------
In the context of a gridsearch, we want to calculate the performance of our algorithm and rank the different
parameter candidates accordingly.
This is what our score function is for.
It gets a pipeline object (**without** results!) and a data point (i.e. a single recording) as input and should
return a some sort of performance metric.
A higher value is always considered better.
If you want to calculate multiple performance measures, you can also return a dictionary of such values.
In any case, the performance for a specific parameter combination in the GridSearch will be calculated as the mean
over all datapoints.
(Note, if you want to change this, you can create custom subclasses of `GaitmapScorer`).

A typical score function will first call `safe_run` (which calls `run` internally) on the pipeline and then
compare the output with some reference.
This reference should be supplied as part of the dataset.

Instead of using a function as scorer (shown here), you can also implement a method called `score` on your pipeline.
Then just pass `None` (which is the default) for the `scoring` parameter in the GridSearch (and other optimizers).
However, a function is usually more flexible.

In this case we compare the calculated stride lists with the reference and identify which strides were correctly
found.
Based on these matches, we calculate the precision, the recall, and the f1-score using some helper functions.

.. GENERATED FROM PYTHON SOURCE LINES 138-151

.. code-block:: default

    from gaitmap.evaluation_utils import evaluate_segmented_stride_list, precision_recall_f1_score


    def score(pipeline: MyPipeline, datapoint: MyDataset):
        # We use the `safe_run` wrapper instead of just run. This is always a good idea.
        pipeline.safe_run(datapoint)
        matches_df = evaluate_segmented_stride_list(
            ground_truth=datapoint.segmented_stride_list_, segmented_stride_list=pipeline.segmented_stride_list_
        )
        assert isinstance(matches_df, pd.DataFrame)
        return precision_recall_f1_score(matches_df)









.. GENERATED FROM PYTHON SOURCE LINES 152-158

The Parameters
--------------
The last step before running the GridSearch, is to select the parameters we want to test for each dataset.
For this, we can directly use sklearn's `ParameterGrid`.

In this example, we will just test two values for the `max_cost` threshold.

.. GENERATED FROM PYTHON SOURCE LINES 158-162

.. code-block:: default

    from sklearn.model_selection import ParameterGrid

    parameters = ParameterGrid({"max_cost": [3, 5]})








.. GENERATED FROM PYTHON SOURCE LINES 163-170

Running the GridSearch
----------------------
Now we have all the pieces to run the GridSearch.
After initializing, we can use `optimize` to run the GridSearch.

.. note:: If the score function returns a dictionary of scores, `rank_scorer` must be set to the name of the score,
          that should be used to decide on the best parameter set.

.. GENERATED FROM PYTHON SOURCE LINES 170-175

.. code-block:: default

    from tpcp.optimize import GridSearch

    gs = GridSearch(pipe, parameters, scoring=score, return_optimized="f1_score")
    gs = gs.optimize(MyDataset())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Parameter Combinations:   0%|          | 0/2 [00:00<?, ?it/s]

    Datapoints:   0%|          | 0/2 [00:00<?, ?it/s]

    Datapoints:  50%|#####     | 1/2 [00:00<00:00,  9.53it/s]
    Datapoints: 100%|##########| 2/2 [00:00<00:00, 11.01it/s]

    Parameter Combinations:  50%|#####     | 1/2 [00:00<00:00,  4.78it/s]

    Datapoints:   0%|          | 0/2 [00:00<?, ?it/s]

    Datapoints: 100%|##########| 2/2 [00:00<00:00, 13.06it/s]
    Datapoints: 100%|##########| 2/2 [00:00<00:00, 13.03it/s]

    Parameter Combinations: 100%|##########| 2/2 [00:00<00:00,  5.31it/s]
    Parameter Combinations: 100%|##########| 2/2 [00:00<00:00,  5.22it/s]




.. GENERATED FROM PYTHON SOURCE LINES 176-179

The main results are stored in `gs_results_`.
It shows the mean performance per parameter combination, the rank for each parameter combination and the
performance for each individual data point (in our case a single recording of one sensor).

.. GENERATED FROM PYTHON SOURCE LINES 179-182

.. code-block:: default

    results = gs.gs_results_
    pd.DataFrame(results)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>agg__precision</th>
          <th>rank__agg__precision</th>
          <th>agg__recall</th>
          <th>rank__agg__recall</th>
          <th>agg__f1_score</th>
          <th>rank__agg__f1_score</th>
          <th>single__precision</th>
          <th>single__recall</th>
          <th>single__f1_score</th>
          <th>data_labels</th>
          <th>debug__score_time</th>
          <th>param__max_cost</th>
          <th>params</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>1.000000</td>
          <td>1</td>
          <td>0.254762</td>
          <td>2</td>
          <td>0.393293</td>
          <td>2</td>
          <td>[1.0, 1.0]</td>
          <td>[0.14285714285714285, 0.36666666666666664]</td>
          <td>[0.25, 0.5365853658536585]</td>
          <td>[(test, left), (test, right)]</td>
          <td>0.206745</td>
          <td>3</td>
          <td>{'max_cost': 3}</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.982759</td>
          <td>2</td>
          <td>0.966667</td>
          <td>1</td>
          <td>0.974576</td>
          <td>1</td>
          <td>[1.0, 0.9655172413793104]</td>
          <td>[1.0, 0.9333333333333333]</td>
          <td>[1.0, 0.9491525423728815]</td>
          <td>[(test, left), (test, right)]</td>
          <td>0.171708</td>
          <td>5</td>
          <td>{'max_cost': 5}</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 183-185

Further, the `optimized_pipeline_` parameter holds an instance of the pipeline initialized with the best parameter
combination.

.. GENERATED FROM PYTHON SOURCE LINES 185-188

.. code-block:: default

    print("Best Para Combi:", gs.best_params_)
    print("Paras of optimized Pipeline:", gs.optimized_pipeline_.get_params())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Best Para Combi: {'max_cost': 5}
    Paras of optimized Pipeline: {'max_cost': 5}




.. GENERATED FROM PYTHON SOURCE LINES 189-193

To run the optmized pipeline, we can directly use the `run`/`safe_run` method on the GridSearch object.
This makes it possible to use the `GridSearch` as a replacement for your pipeline object with minimal code changes.

If you would try to call `run`/`safe_run` (or `score` for that matter), before the optimization, an error is raised.

.. GENERATED FROM PYTHON SOURCE LINES 193-195

.. code-block:: default

    segmented_stride_list = gs.safe_run(dataset[0]).segmented_stride_list_
    segmented_stride_list





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>start</th>
          <th>end</th>
        </tr>
        <tr>
          <th>s_id</th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>364</td>
          <td>584</td>
        </tr>
        <tr>
          <th>1</th>
          <td>584</td>
          <td>802</td>
        </tr>
        <tr>
          <th>2</th>
          <td>802</td>
          <td>1023</td>
        </tr>
        <tr>
          <th>3</th>
          <td>1023</td>
          <td>1242</td>
        </tr>
        <tr>
          <th>4</th>
          <td>1242</td>
          <td>1458</td>
        </tr>
        <tr>
          <th>5</th>
          <td>1458</td>
          <td>1672</td>
        </tr>
        <tr>
          <th>6</th>
          <td>1672</td>
          <td>1887</td>
        </tr>
        <tr>
          <th>7</th>
          <td>1887</td>
          <td>2104</td>
        </tr>
        <tr>
          <th>8</th>
          <td>2104</td>
          <td>2327</td>
        </tr>
        <tr>
          <th>9</th>
          <td>2327</td>
          <td>2546</td>
        </tr>
        <tr>
          <th>10</th>
          <td>2546</td>
          <td>2773</td>
        </tr>
        <tr>
          <th>11</th>
          <td>2773</td>
          <td>2998</td>
        </tr>
        <tr>
          <th>12</th>
          <td>2998</td>
          <td>3231</td>
        </tr>
        <tr>
          <th>13</th>
          <td>3231</td>
          <td>3453</td>
        </tr>
        <tr>
          <th>14</th>
          <td>3934</td>
          <td>4163</td>
        </tr>
        <tr>
          <th>15</th>
          <td>4163</td>
          <td>4382</td>
        </tr>
        <tr>
          <th>16</th>
          <td>4382</td>
          <td>4603</td>
        </tr>
        <tr>
          <th>17</th>
          <td>4603</td>
          <td>4822</td>
        </tr>
        <tr>
          <th>18</th>
          <td>4822</td>
          <td>5043</td>
        </tr>
        <tr>
          <th>19</th>
          <td>5043</td>
          <td>5267</td>
        </tr>
        <tr>
          <th>20</th>
          <td>5267</td>
          <td>5489</td>
        </tr>
        <tr>
          <th>21</th>
          <td>5489</td>
          <td>5713</td>
        </tr>
        <tr>
          <th>22</th>
          <td>5713</td>
          <td>5936</td>
        </tr>
        <tr>
          <th>23</th>
          <td>5936</td>
          <td>6167</td>
        </tr>
        <tr>
          <th>24</th>
          <td>6167</td>
          <td>6395</td>
        </tr>
        <tr>
          <th>25</th>
          <td>6395</td>
          <td>6628</td>
        </tr>
        <tr>
          <th>26</th>
          <td>6628</td>
          <td>6858</td>
        </tr>
        <tr>
          <th>27</th>
          <td>6858</td>
          <td>7091</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  3.545 seconds)

**Estimated memory usage:**  10 MB


.. _sphx_glr_download_auto_examples_datasets_and_pipelines_gridsearch.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: gridsearch.py <gridsearch.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: gridsearch.ipynb <gridsearch.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_