Introduction to the InSituExperiment class

Introduction to the InSituExperiment class#

Background#

../../_images/experiment_levels.jpg

In addition to the sample, cellular, and subcellular levels, a spatial transcriptomics dataset typically includes an experiment level that contains information about clinical and experimental cohorts, conditions, or treatments. Most current computational frameworks lack functionalities to incorporate this level into the analysis. To address this, InSituPy introduces the InSituExperiment class, which manages spatial transcriptomics data and the corresponding metadata of multiple samples simultaneously.

Structure#

../../_images/insituexperiment_structure.jpg

An InSituExperiment object consists of multiple InSituData objects paired with their corresponding metadata.

## The following code ensures that all functions and init files are reloaded before executions.
%load_ext autoreload
%autoreload 2

from pathlib import Path
from insitupy import InSituData, InSituExperiment, CACHE

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

Load Xenium data into InSituData object#

Now the Xenium data can be parsed by providing the data path to the InSituPy project folder.

insitupy_project = Path(CACHE / "out/demo_insitupy_project")
xd = InSituData.read(insitupy_project)
xd.load_all(skip="transcripts")
xd

InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		C:\Users\ge37voy\.cache\InSituPy\out\demo_insitupy_project
Metadata file:	.ispy
    ➤ images
       nuclei:	(25778, 35416)
       CD20:	(25778, 35416)
       HER2:	(25778, 35416)
       HE:	(25778, 35416, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 156447 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       TestKey:	9 annotations, 2 classes ('TestClass','points') ✔
       demo:	4 annotations, 1 class ('None') ✔
       demo2:	5 annotations, 1 class ('None') ✔
       demo3:	7 annotations, 1 class ('None') ✔
       Demo:	28 annotations, 2 classes ('Tumor cells','Stroma') ✔
    ➤ regions
       demo_regions:	3 regions, 3 classes ('Region1','Region2','Region3') ✔
       TMA:	6 regions, 6 classes ('B-2','A-3','B-1','B-3','A-1','A-2') ✔
       Demo:	3 regions, 3 classes ('Region 1','Region 3','Region 2') ✔

Create InSituExperiment#

../../_images/insituexperiment_generation.jpg

Options to generate an InSituExperiment object

Method 1: Manually add InSituData objects#

exp = InSituExperiment()
exp.add(
    data=xd,
    metadata={
        "slide_id": xd.slide_id,
        "sample_id": xd.sample_id,
        "patient": "A"
    }
    )

exp

InSituExperiment with 1 samples:
           uid  CITAR slide_id    sample_id patient
0     9d31447e  ++-++  0001879  Replicate 1       A

In the same way also other datasets can be added. For demonstration purposes, we just add the same dataset again and change the metadata.

exp.add(
    data=xd,
    metadata={
        "slide_id": xd.slide_id,
        "sample_id": xd.sample_id,
        "patient": "A",
        "therapy": "drugB"
    }
    )

exp

InSituExperiment with 2 samples:
           uid  CITAR slide_id    sample_id patient therapy
0     9d31447e  ++-++  0001879  Replicate 1       A     NaN
1     f29fc2f4  ++-++  0001879  Replicate 1       A   drugB

Method 2: From config file#

As config file either a csv file or an excel file can be used.

Example of a valid configuration file:

directory

experiment_name

date

patient

/path/to/dataset1

Experiment 1

2023-09-01

Patient A

/path/to/dataset2

Experiment 2

2023-09-02

Patient B

 
import pandas as pd
from insitupy import CACHE

# Define the data
data = {
    "directory": [f"{CACHE}\\out\\demo_insitupy_project", f"{CACHE}\\out\\demo_insitupy_project"],
    "patient": ["A", "B"],
    "therapy": ["drugA", "drugB"]
}

# Create a DataFrame
config_df = pd.DataFrame(data)

# Write the DataFrame to a CSV file
config_df.to_csv('../../demo_data/demo_experiment/insituexperiment_config.csv', index=False)

exp = InSituExperiment.from_config(config_path="../../demo_data/demo_experiment/insituexperiment_config.csv")

exp

InSituExperiment with 2 samples:
           uid  CITAR slide_id    sample_id patient therapy
0     c6dd1bcc  -----  0001879  Replicate 1       A   drugA
1     8afcb324  -----  0001879  Replicate 1       B   drugB

exp.load_all()

exp

InSituExperiment with 2 samples:
           uid  CITAR slide_id    sample_id patient therapy
0     c6dd1bcc  +++++  0001879  Replicate 1       A   drugA
1     8afcb324  +++++  0001879  Replicate 1       B   drugB

Method 3: From regions#

We can also use regions from an InSituData object to split the data into separate datasets and create an InSituExperiment from them. This can be used to select the most interesting regions and focus on them for the analysis or to split a TMA dataset into separate datasets for each core.

exp = InSituExperiment.from_regions(
    data=xd, region_key="TMA"
)

A-1
A-2
A-3
B-1
B-2
B-3

exp

InSituExperiment with 6 samples:
           uid  CITAR slide_id    sample_id region_key region_name
0     21504fb3  ++-++  0001879  Replicate 1        TMA         A-1
1     c55f6f95  ++-++  0001879  Replicate 1        TMA         A-2
2     e741e9b0  ++-++  0001879  Replicate 1        TMA         A-3
3     f3d39873  ++-++  0001879  Replicate 1        TMA         B-1
4     25ad7cd8  ++-++  0001879  Replicate 1        TMA         B-2
5     b9718fd2  ++-++  0001879  Replicate 1        TMA         B-3

# visualize data
viewer = exp.show(3)

Example screenshot of data from one of the cropped cores:

../../_images/TMA_example.jpg

Plot overview of metadata and QC metrics#

from insitupy.plotting import plot_overview

plot_overview(exp)
../../_images/2b93d8e6f5a628b7130ec683da523953486b008c6946a880d85ef4a97b58d096.png 
plot_overview(exp, cells_layer="main", colums_to_plot=["uid", "slide_id", "sample_id", "region_name", "region_key"], index=True)
../../_images/050bb95c672b7369d8e29ee0072656b790603bd8cdf39fd4482a17bec3d95d09.png

Iterate through InSituExperiment using .iterdata() method#

for metadata, data in exp.iterdata():
    print(f"Metadata:\n{metadata[:3]}\nData:\n{data}")

Metadata:
uid             21504fb3
slide_id         0001879
sample_id    Replicate 1
Name: 0, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4706)
       CD20:	(4706, 4706)
       HER2:	(4706, 4706)
       HE:	(4706, 4706, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 4847 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       demo:	4 annotations, 2 classes ('Stroma','Tumor cells') 
       Demo:	4 annotations, 2 classes ('Stroma','Tumor cells') 
    ➤ regions
       TMA:	1 regions, 1 class ('A-1') 
       Demo:	1 regions, 1 class ('Region 1') 
Metadata:
uid             c55f6f95
slide_id         0001879
sample_id    Replicate 1
Name: 1, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4705)
       CD20:	(4706, 4705)
       HER2:	(4706, 4705)
       HE:	(4706, 4705, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 1766 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       TestKey:	1 annotations, 1 class ('TestClass') 
    ➤ regions
       TMA:	1 regions, 1 class ('A-2') 
Metadata:
uid             e741e9b0
slide_id         0001879
sample_id    Replicate 1
Name: 2, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4705)
       CD20:	(4706, 4705)
       HER2:	(4706, 4705)
       HE:	(4706, 4705, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 1807 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       
    ➤ regions
       TMA:	1 regions, 1 class ('A-3') 
Metadata:
uid             f3d39873
slide_id         0001879
sample_id    Replicate 1
Name: 3, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4706)
       CD20:	(4706, 4706)
       HER2:	(4706, 4706)
       HE:	(4706, 4706, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 3569 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       demo2:	1 annotations, 1 class ('None') 
    ➤ regions
       TMA:	1 regions, 1 class ('B-1') 
Metadata:
uid             25ad7cd8
slide_id         0001879
sample_id    Replicate 1
Name: 4, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4706)
       CD20:	(4706, 4706)
       HER2:	(4706, 4706)
       HE:	(4706, 4706, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 2301 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       demo:	1 annotations, 1 class ('Stroma') 
       demo2:	1 annotations, 1 class ('None') 
       Demo:	1 annotations, 1 class ('Stroma') 
    ➤ regions
       demo_regions:	1 regions, 1 class ('Region3') 
       TMA:	1 regions, 1 class ('B-2') 
       Demo:	1 regions, 1 class ('Region 2') 
Metadata:
uid             b9718fd2
slide_id         0001879
sample_id    Replicate 1
Name: 5, dtype: object
Data:
InSituData
Method:		Xenium
Slide ID:	0001879
Sample ID:	Replicate 1
Path:		None
Metadata file:	.ispy
    ➤ images
       nuclei:	(4706, 4706)
       CD20:	(4706, 4706)
       HER2:	(4706, 4706)
       HE:	(4706, 4706, 3)
    ➤ cells
       MultiCellData with main layer 'main'
           matrix
               AnnData object with n_obs × n_vars = 3405 × 297
               obs: 'transcript_counts', 'control_probe_counts', 'control_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'n_genes_by_counts', 'n_genes', 'leiden', 'cell_type_dc', 'cell_type_dc_sub', 'cell_type_tacco', 'cell_type_publ'
               var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'pct_dropout_by_counts', 'total_counts', 'n_cells'
               uns: 'cell_type_dc_colors', 'cell_type_dc_sub', 'cell_type_dc_sub_colors', 'cell_type_publ_colors', 'cell_type_tacco_colors', 'counts_location', 'leiden', 'leiden_colors', 'log1p', 'neighbors', 'pca', 'umap'
               obsm: 'OT', 'X_pca', 'X_umap', 'annotations', 'ora_estimate', 'ora_pvals', 'regions', 'spatial'
               varm: 'OT', 'PCs'
               layers: 'counts', 'norm_counts'
               obsp: 'connectivities', 'distances'
           boundaries
               BoundariesData object with 2 entries:
                   cells
                   nuclei
    ➤ annotations
       demo:	3 annotations, 2 classes ('Stroma','Tumor cells') 
       demo2:	1 annotations, 1 class ('None') 
       Demo:	3 annotations, 2 classes ('Stroma','Tumor cells') 
    ➤ regions
       TMA:	1 regions, 1 class ('B-3') 
       Demo:	1 regions, 1 class ('Region 3')

Add new metadata#

In the following section different scenarios for adding new metadata are shown.

exp1 = InSituExperiment.from_regions(
    data=xd, region_key="demo_regions"
)

Region1
Region2
Region3

exp1

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id    region_key region_name
0     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1
1     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2
2     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3

exp2 = exp1.copy()
exp2.append_metadata(
    new_metadata="../../demo_data/demo_experiment/insituexperiment_new_metadata.csv",
    by="region_name", overwrite=True
)

exp2

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id region_name therapy   organ      test region_key
0     3493cd71  ++-++  0001879  Replicate 1     Region1   drugC   liver  negative       key1
1     b750995b  ++-++  0001879  Replicate 1     Region2   drugD    lung      test       key3
2     6c7ba21d  ++-++  0001879  Replicate 1     Region3   drugE  spleen  positive       key2

exp3 = exp1.copy()
exp3.append_metadata(
    new_metadata="../../demo_data/demo_experiment/insituexperiment_new_metadata.csv",
    by="region_name", overwrite=False
)

exp3

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id    region_key region_name   organ      test therapy
0     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1   liver  negative   drugC
1     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2    lung      test   drugD
2     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3  spleen  positive   drugE

exp4 = exp1.copy()
exp4.append_metadata(
    new_metadata="../../demo_data/demo_experiment/insituexperiment_new_metadata2.csv",
    by="region_name", overwrite=False
)

exp4

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id    region_key region_name   organ      test therapy
0     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1   liver  negative   drugC
1     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2     NaN       NaN     NaN
2     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3  spleen  positive   drugE

exp5 = exp1.copy()
exp5.append_metadata(
    new_metadata="../../demo_data/demo_experiment/insituexperiment_new_metadata2.csv",
    by="region_name", overwrite=True
)

exp5

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id region_name therapy   organ      test region_key
0     3493cd71  ++-++  0001879  Replicate 1     Region1   drugC   liver  negative       key1
1     b750995b  ++-++  0001879  Replicate 1     Region2     NaN     NaN       NaN        NaN
2     6c7ba21d  ++-++  0001879  Replicate 1     Region3   drugE  spleen  positive       key2

Concatenate multiple InSituExperiment objects

exp

InSituExperiment with 6 samples:
           uid  CITAR slide_id    sample_id region_key region_name
0     21504fb3  ++-++  0001879  Replicate 1        TMA         A-1
1     c55f6f95  ++-++  0001879  Replicate 1        TMA         A-2
2     e741e9b0  ++-++  0001879  Replicate 1        TMA         A-3
3     f3d39873  ++-++  0001879  Replicate 1        TMA         B-1
4     25ad7cd8  ++-++  0001879  Replicate 1        TMA         B-2
5     b9718fd2  ++-++  0001879  Replicate 1        TMA         B-3

exp1

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id    region_key region_name
0     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1
1     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2
2     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3

exp_concat = InSituExperiment.concat(
    objs={
        "exp_TMA": exp,
        "exp_demo_regions": exp1
    },
    new_col_name="name"
    )

exp_concat

InSituExperiment with 9 samples:
           uid  CITAR slide_id    sample_id    region_key region_name            name
0     21504fb3  ++-++  0001879  Replicate 1           TMA         A-1         exp_TMA
1     c55f6f95  ++-++  0001879  Replicate 1           TMA         A-2         exp_TMA
2     e741e9b0  ++-++  0001879  Replicate 1           TMA         A-3         exp_TMA
3     f3d39873  ++-++  0001879  Replicate 1           TMA         B-1         exp_TMA
4     25ad7cd8  ++-++  0001879  Replicate 1           TMA         B-2         exp_TMA
5     b9718fd2  ++-++  0001879  Replicate 1           TMA         B-3         exp_TMA
6     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1  exp_demo_re...
7     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2  exp_demo_re...
8     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3  exp_demo_re...

exp_concat = InSituExperiment.concat(
    objs=[exp, exp1])

exp_concat

InSituExperiment with 9 samples:
           uid  CITAR slide_id    sample_id    region_key region_name
0     21504fb3  ++-++  0001879  Replicate 1           TMA         A-1
1     c55f6f95  ++-++  0001879  Replicate 1           TMA         A-2
2     e741e9b0  ++-++  0001879  Replicate 1           TMA         A-3
3     f3d39873  ++-++  0001879  Replicate 1           TMA         B-1
4     25ad7cd8  ++-++  0001879  Replicate 1           TMA         B-2
5     b9718fd2  ++-++  0001879  Replicate 1           TMA         B-3
6     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1
7     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2
8     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3

Indexing and selection of data within InSituExperiment#

The InSituExperiment class allows simple selection and filtering of data using indices or True/False masks.

Example 1: Select datasets by index#

exp_indexed = exp_concat[:3]

exp_indexed

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id region_key region_name
0     21504fb3  ++-++  0001879  Replicate 1        TMA         A-1
1     c55f6f95  ++-++  0001879  Replicate 1        TMA         A-2
2     e741e9b0  ++-++  0001879  Replicate 1        TMA         A-3

exp_indexed.show(2)

Viewer(camera=Camera(center=(0.0, 286.875, 351.475), zoom=0.28777131886299395, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(1.0, 1.0), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=2, ndisplay=2, last_used=0, range=((0.0, 573.9625, 0.2125), (0.0, 703.1625, 0.2125)), current_step=(1350, 1654), order=(0, 1), axis_labels=('0', '1')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'nuclei' at 0x235aa8b1790>, <Image layer 'CD20' at 0x235e53fa6d0>, <Image layer 'HER2' at 0x235e5414850>, <Image layer 'HE' at 0x235e7cc3430>], help='use <2> for transform', status='Ready', tooltip=Tooltip(visible=False, text=''), theme='dark', title='0001879: Replicate 1', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x000002350FA8B8B0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={})

Also a list of indices can be used for this.

exp_concat[[0,1,4]]

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id region_key region_name
0     21504fb3  ++-++  0001879  Replicate 1        TMA         A-1
1     c55f6f95  ++-++  0001879  Replicate 1        TMA         A-2
2     25ad7cd8  ++-++  0001879  Replicate 1        TMA         B-2

Example 2: Select only datasets with a “1” in the region name using a mask#

filtering_mask = exp_concat.metadata["region_name"].str.contains("1", na=False) # NaN values are filled with False
exp_filtered = exp_concat[filtering_mask]

Example 3: Query data#

The .query() function can be used to the pandas.DataFrame.query function.

exp_concat.query(
    criteria="region_key == 'demo_regions'"
)

InSituExperiment with 3 samples:
           uid  CITAR slide_id    sample_id    region_key region_name
0     3493cd71  ++-++  0001879  Replicate 1  demo_regions     Region1
1     b750995b  ++-++  0001879  Replicate 1  demo_regions     Region2
2     6c7ba21d  ++-++  0001879  Replicate 1  demo_regions     Region3

Save InSituExperiment#

exp_concat.saveas(CACHE / "out/test_insituexperiment", overwrite=True)

Reload InSituExperiment#

exp_reloaded = InSituExperiment.read(CACHE / "out/test_insituexperiment/")

exp_reloaded.load_all()

exp_reloaded

InSituExperiment with 5 samples:
           uid  CITAR  slide_id    sample_id patient therapy    region_key region_name
0     fbc3579c  -----      1879  Replicate 1       A   drugA           NaN         NaN
1     7168bb60  -----      1879  Replicate 1       B   drugB           NaN         NaN
2     e41835c6  ++-++      1879  Replicate 1     NaN     NaN  demo_regions     Region1
3     b18057d7  ++-++      1879  Replicate 1     NaN     NaN  demo_regions     Region2
4     4668ae9b  ++-++      1879  Replicate 1     NaN     NaN  demo_regions     Region3

exp_reloaded.show(2)

Viewer(camera=Camera(center=(0.0, 286.875, 351.475), zoom=0.28777131886299395, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(1.0, 1.0), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=2, ndisplay=2, last_used=0, range=((0.0, 573.9625, 0.2125), (0.0, 703.1625, 0.2125)), current_step=(1350, 1654), order=(0, 1), axis_labels=('0', '1')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'nuclei' at 0x235f9e93760>, <Image layer 'CD20' at 0x235fa107340>, <Image layer 'HER2' at 0x235fa2d9970>, <Image layer 'HE' at 0x236000ed8e0>], help='use <2> for transform', status='Ready', tooltip=Tooltip(visible=False, text=''), theme='dark', title='0001879: Replicate 1', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x000002350FA8B8B0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={})
Contents