Analyses of the IPRGC re-processed samples.

1 TODOs

• Check this for correctness.
• Reorganize it
• GSVA (Hänzelmann, Castelo, and Guinney (2013))
• Consider embedding some/many/all of the Excel outputs into the html output via xfun::embed_dir(‘excel/’)

2 Meeting with Theresa

Previous papers did not do an explicit subtraction, instead just compared to WT and kept the genes which are > in delta/het vs. wt. There are multiple ways to deal with this and that query has not yet been defined. Later, Theresa came to the conclusion that the subtraction method is not appropriate.

3 Introduction

In this document I hope to explore the freshly processed samples and perform some comparisons to see that we have the expected similarities and differences from the prior analysis performed by Theresa.

There is one way in which I expect any/all of these analyses to be explicitly different: this should include the changes produced by April’s renaming of some samples.

My intention is to produce a sample sheet which includes one column with non-umi-deduplicated results and one with deduplicated results. With the exception of the previous point, I hope that the first will be identical (or at least very close to identical) to Theresa’s result while the second I expect will be subtly different – but I am hoping subtly enough that it will not significantly change the interpretation but be a little more precise.

Lets see! I need therefore to make a change to my metadata gathering function to include the umi deduplicated result. I am thinking therefore to create a separate specification for umi-barcoded samples because looking through the logs for umi stuff when they are not used will be too much of a pain…

3.1 Small random reminder

I have a couple pictures of RPL22 to help me remember the experimental design:

• The human ribosome with RPL22 in red:
• The mouse ribosome with RPL22 in green at the center:

That second picture came from: (Li et al. (2022))

4 A note about implementation

I would like to improve this document by comparing/contrasting the methodologies performed by other groups and those performed by me in it. I never fully appreciated the suite of computational methods applied by previous groups when examining TRAP data; I instead simply followed Theresa’s notebook without considering other possibilities.

I therefore spent a little time stepping through her thesis and pulling out the relevant papers in the hopes of learning these various methods. I should therefore be able soon to compare/contrast the various methods employed by other labs in addition to copying Theresa’s logic.

4.1 The following block cannot work in the container

The following block assumes the full tree of preprocessed data with the logs from the trimmer, mapping, umi deduplication, counting, etc. As a result it cannot work in the container which has only the various count tables.

As a result, I am including a copy of this sheet after running the following block in my working tree. I suppose for the moment you will have to trust that it worked. (for right now, when testing out this container, I am just sending the R working directory to my tree for this block, then moving it back.

I will need to manually edit one column though, the symlink column from Theresa has a series of paths which do not work in the container.

umi_spec <- make_rnaseq_spec(umi = TRUE)
iprgc_2022_meta <- gather_preprocessing_metadata("sample_sheets/20240606_only_umd_sequenced.xlsx",
                                                 spec = umi_spec, species = "mm39_112", verbose = FALSE,
                                                 basedir = "preprocessing/umd_sequenced")
colnames(iprgc_2022_meta[["new_meta"]])
head(iprgc_2022_meta[["new_meta"]])

sample_sheet <- "sample_sheets/20240606_only_umd_sequenced_modified.xlsx"

From this point on, I am hoping/intending to pull liberally from Theresa’s notebook with a diversion to compare the three datasets:

• Pre-April renaming: E.g. Theresa’s current dataset
• Post renaming: Unless I am mistaken, this should be very similar to the above.
• Post deduplication: Given what I saw from the extracted logs in the sample sheet, I expect this to be similar but not identical to the previous two.

Lets find out! But first, annotations!

5 Annotation data

I am pulling this from Theresa’s anxontrapR_pipeline.Rmd, primarily because it looks similar to the other documents, but was modified more recently. I will change it slightly, primarily because I grabbed a new mmusculus assembly and therefore I will pull the mmusculus annotations from a specific biomart (Smedley et al. (2009)) archive that should match it.

A note from the future: multiple ensembl archive servers have been taken offline since last I ran this. Let us see if Feb. 2023 still works.

mm_annot <- load_biomart_annotations(species = "mmusculus", year = "2023", month = "02")

## Using mart: ENSEMBL_MART_ENSEMBL from host: feb2023.archive.ensembl.org.

## Successfully connected to the mmusculus_gene_ensembl database.

## Finished downloading ensembl gene annotations.

## Finished downloading ensembl structure annotations.

## symbol columns is null, pattern matching 'symbol' and taking the first.

## Found 3 potential symbol columns, using the first:hgnc_symbol.

## Including symbols, there are 57010 vs the 149443 gene annotations.

## Not dropping haplotype chromosome annotations, set drop_haplotypes = TRUE if this is bad.

## Saving annotations to mmusculus_biomart_annotations.rda.

## Finished save().

mm_annot <- mm_annot[["annotation"]]
mm_annot[["txid"]] <- paste0(mm_annot[["ensembl_transcript_id"]], ".", mm_annot[["version"]])
rownames(mm_annot) <- make.names(mm_annot[["ensembl_gene_id"]], unique=TRUE)
tx_gene_map <- mm_annot[, c("txid", "ensembl_gene_id")]

6 Hisat2 summarizedExperiments

The primary difference between my block and Theresa’s are:

1. I am pulling the metadata directly from the gather_preprocessing_metadata() above.
2. I am using the column ‘symlink’ which is just a copy of the existing ‘file’ column with a small change so I can load it from my directory without having to copy everything.
3. I am using the ensembl genome release 39, version 112 and so pulled a somewhat newer copy of the annotation data.
4. The original is named ‘v1’, followed by ‘v2’ and ‘v3’ for the other two treatments I performed.

6.1 Color choices and reused parameters

Given that we are excluding a bunch of the older samples, the set of colors I expect to find is different; so I will make explicit here the various colors used to denote location/genotype/time/etc.

April turned me onto this website ‘paletton.com’ for this kind of stuff and I will try and pick out palettes which basically match what I am getting with the original colors.

color_choices <- list(
  "all" = list(
    "p08_het_dlgn" = "#E7298A",
    "p15_het_dlgn" = "#E7298A",
    "p08_het_retina" = "#238B45",
    "p15_het_retina" = "#238B45",
    "p08_het_scn" = "#4292C6",
    "p15_het_scn" = "#4292C6",
    "p08_ko_dlgn" = "#C994C7",
    "p15_ko_dlgn" = "#C994C7",
    "p08_ko_retina" = "#74c476",
    "p15_ko_retina" = "#74c476",
    "p08_ko_scn" = "#9BCAE1",
    "p15_ko_scn" = "#9BCAE1",
    "p08_wt_dlgn" = "#980043",
    "p15_wt_dlgn" = "#980043",
    "p08_wt_retina" = "#004008",
    "p15_wt_retina" = "#004008",
    "p08_wt_scn" = "#08519C",
    "p15_wt_scn" = "#08519C",
    "p60_wt_dlgn" = "#333333",
    "p60_wt_retina" = "#222222",
    "p60_wt_scn" = "#111111"),
  "geno_loc" = list(
    "het_dlgn" = "#E7298A",
    "het_retina" = "#238B45",
    "het_scn" = "#4292C6",
    "ko_dlgn" = "#C994C7",
    "ko_retina" = "#74c476",
    "ko_scn" = "#9BCAE1",
    "wt_dlgn" = "#980043",
    "wt_retina" = "#004008",
    "wt_scn" = "#08519C"),
  "location" = list(
    "retina" = "#004008",
    "dlgn" = "#980043",
    "scn" = "#08519C"),
  "genotype" = list(
    "wt" = "#74c476",
    "het" = "#238B45",
    "ko" = "#006D2C"),
  "time" = list(
    "p08" = "#5E104B",
    "p15" = "#4E9231"))
label_column <- "mgi_symbol" ## Set the column used to extract gene symbols rather than ENSG.....
colors <- color_choices[["geno_loc"]]

There is one noteworthy sample: iprgc_103, it was effectively replaced when April renamed the samples and so exists in the v1 data, but not v2/v3; they instead have the newly named samples which I called iprgc_123 to iprgc_130. As a result, I copied the annotations for iprgc_123 to my column so that there is no discrepency in terms of genotype/location/time.

6.2 The original count tables

At the moment I have not included the original counts in this container because we made some changes to the mapping strategy and also found that a couple samples were mixed up in sequencing; as a result I documented all of the changes in the sample sheets and preprocessing documents and excluded the original files.

This is also why some columns in the sample sheet have suffixes like ‘adh’ and ‘atb’, those denote from whom the relevant metadata columns came from.

mm38_hisat_v1 <- create_se(sample_sheet,
                           gene_info = mm_annot,
                           file_column = "symlink") %>%
  set_conditions(fact = "geno_loc_atb") %>%
  set_batches(fact = "time_atb") %>%
  set_colors(color_choices[["geno_loc"]])
mm38_hisat_v1

6.3 Recounted tables and the deduplicated result

In the following I make two more versions of the data, one remapped with the changes to the sample identities, and one with deduplication applied.

mm38_hisat_v2 <- create_se(sample_sheet, gene_info = mm_annot,
                           file_column = "hisat_count_table") %>%
  set_conditions(fact = "geno_loc_atb") %>%
  set_batches(fact = "time_atb") %>%
  set_colors(color_choices[["geno_loc"]])

## Reading the sample metadata.

## Checking the state of the condition column.

## Checking the state of the batch column.

## Checking the condition factor.

## The sample definitions comprises: 69 rows(samples) and 76 columns(metadata fields).

## Warning in create_se(sample_sheet, gene_info = mm_annot, file_column =
## "hisat_count_table"): Some samples were removed when cross referencing the
## samples against the count data.

## Matched 25404 annotations and counts.

## Some annotations were lost in merging, setting them to 'undefined'.

## The final summarized experiment has 25425 rows and 76 columns.

## The numbers of samples by condition are:

## 
##   het_dlgn het_retina    het_scn    ko_dlgn  ko_retina     ko_scn    wt_dlgn 
##          7          7          7          6          6          6         11 
##  wt_retina     wt_scn 
##         11          7

## The number of samples by batch are:

## 
## p08 p15 p60 
##  31  34   3

mm38_hisat_v2

## class: SummarizedExperiment 
## dim: 25425 68 
## metadata(7): notes title ... study researcher
## assays(1): ''
## rownames(25425): ENSMUSG00000000001 ENSMUSG00000000003 ...
##   ENSMUSG00001074846 ENSMUSG00002076083
## rowData names(13): ensembl_gene_id ensembl_transcript_id ...
##   hgnc_symbol txid
## colnames(68): iprgc_62 iprgc_63 ... iprgc_129 iprgc_130
## colData names(76): rownames sampleid ... umi_dedup_mean_umi_per_pos
##   umi_dedup_max_umi_per_pos

mm38_hisat_v3 <- create_se(sample_sheet, gene_info = mm_annot,
                           file_column = "umi_dedup_output_count") %>%
  set_conditions(fact = "geno_loc_atb") %>%
  set_batches(fact = "time_atb") %>%
  set_colors(color_choices[["geno_loc"]])

## Reading the sample metadata.

## Checking the state of the condition column.

## Checking the state of the batch column.

## Checking the condition factor.

## The sample definitions comprises: 69 rows(samples) and 76 columns(metadata fields).

## Warning in create_se(sample_sheet, gene_info = mm_annot, file_column =
## "umi_dedup_output_count"): Some samples were removed when cross referencing the
## samples against the count data.

## Matched 25404 annotations and counts.

## Some annotations were lost in merging, setting them to 'undefined'.

## The final summarized experiment has 25425 rows and 76 columns.

## The numbers of samples by condition are:

## 
##   het_dlgn het_retina    het_scn    ko_dlgn  ko_retina     ko_scn    wt_dlgn 
##          7          7          7          6          6          6         11 
##  wt_retina     wt_scn 
##         11          7

## The number of samples by batch are:

## 
## p08 p15 p60 
##  31  34   3

mm38_hisat_v3

## class: SummarizedExperiment 
## dim: 25425 68 
## metadata(7): notes title ... study researcher
## assays(1): ''
## rownames(25425): ENSMUSG00000000001 ENSMUSG00000000003 ...
##   ENSMUSG00001074846 ENSMUSG00002076083
## rowData names(13): ensembl_gene_id ensembl_transcript_id ...
##   hgnc_symbol txid
## colnames(68): iprgc_62 iprgc_63 ... iprgc_129 iprgc_130
## colData names(76): rownames sampleid ... umi_dedup_mean_umi_per_pos
##   umi_dedup_max_umi_per_pos

all_fact <- paste0(colData(mm38_hisat_v3)[["time_atb"]], "_",
                   colData(mm38_hisat_v3)[["geno_loc_atb"]])
colData(mm38_hisat_v3)[["time_geno_loc"]] <- all_fact

Note the end of the previous block, I created a factor out of the combination of time, genotype, and location. In a future invocation of this notebook, I will change the pairwise comparisons to add each of these three factors to the statistical model instead of this. The code to do that is not quite ready yet.

7 Non-zero Counts per Sample

Let’s look at the number of non-zero genes for all samples versus the coverage.

As above, this does not get run because I did not copy the count tables.

v1_nonzero <- plot_nonzero(mm38_hisat_v1)
v1_nonzero

But these do!

plot_legend(mm38_hisat_v2)

## The colors used in the expressionset are: #004008, #08519C, #238B45, #4292C6, #74c476, #980043, #9BCAE1, #C994C7, #E7298A.

v2_nonzero  <- plot_nonzero(mm38_hisat_v2, y_intercept = 0.65)

## The following samples have less than 16526.25 genes.

##  [1] "iprgc_62"  "iprgc_63"  "iprgc_64"  "iprgc_66"  "iprgc_67"  "iprgc_68" 
##  [7] "iprgc_70"  "iprgc_71"  "iprgc_72"  "iprgc_73"  "iprgc_74"  "iprgc_75" 
## [13] "iprgc_77"  "iprgc_78"  "iprgc_80"  "iprgc_81"  "iprgc_82"  "iprgc_83" 
## [19] "iprgc_84"  "iprgc_85"  "iprgc_86"  "iprgc_87"  "iprgc_88"  "iprgc_89" 
## [25] "iprgc_90"  "iprgc_91"  "iprgc_92"  "iprgc_93"  "iprgc_94"  "iprgc_95" 
## [31] "iprgc_96"  "iprgc_97"  "iprgc_98"  "iprgc_100" "iprgc_102" "iprgc_104"
## [37] "iprgc_105" "iprgc_106" "iprgc_107" "iprgc_108" "iprgc_110" "iprgc_111"
## [43] "iprgc_112" "iprgc_113" "iprgc_114" "iprgc_115" "iprgc_117" "iprgc_118"
## [49] "iprgc_121" "iprgc_123" "iprgc_124" "iprgc_125" "iprgc_126" "iprgc_127"
## [55] "iprgc_128" "iprgc_129" "iprgc_130"

## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.

## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.

## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## i Please use `linewidth` instead.
## i The deprecated feature was likely used in the hpgltools package.
##   Please report the issue to the authors.
## This warning is displayed once per session.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

v2_nonzero

## $plot

## 
## $table
##                  id nonzero_genes    cpm  condition batch   color     label
## iprgc_62   iprgc_62         14542  5.643   het_dlgn   p15 #E7298A  iprgc_62
## iprgc_63   iprgc_63         13692  3.717   het_dlgn   p15 #E7298A  iprgc_63
## iprgc_64   iprgc_64         16456 12.978 het_retina   p15 #238B45  iprgc_64
## iprgc_65   iprgc_65         16881 14.697 het_retina   p15 #238B45  iprgc_65
## iprgc_66   iprgc_66         15388  8.272    het_scn   p15 #4292C6  iprgc_66
## iprgc_67   iprgc_67         15376  7.908    het_scn   p15 #4292C6  iprgc_67
## iprgc_68   iprgc_68         14030  4.603    ko_dlgn   p15 #C994C7  iprgc_68
## iprgc_69   iprgc_69         17005 18.775  ko_retina   p15 #74c476  iprgc_69
## iprgc_70   iprgc_70         14830  5.478     ko_scn   p15 #9BCAE1  iprgc_70
## iprgc_71   iprgc_71         14950  6.549    wt_dlgn   p15 #980043  iprgc_71
## iprgc_72   iprgc_72         15336  7.358    wt_dlgn   p15 #980043  iprgc_72
## iprgc_73   iprgc_73         15213  7.125    wt_dlgn   p15 #980043  iprgc_73
## iprgc_74   iprgc_74         16163 15.075  wt_retina   p15 #004008  iprgc_74
## iprgc_75   iprgc_75         16449 14.804  wt_retina   p15 #004008  iprgc_75
## iprgc_76   iprgc_76         17083 22.572  wt_retina   p15 #004008  iprgc_76
## iprgc_77   iprgc_77         15165  5.038     wt_scn   p15 #08519C  iprgc_77
## iprgc_78   iprgc_78         15305  8.734    wt_dlgn   p60 #980043  iprgc_78
## iprgc_79   iprgc_79         16818 15.018  wt_retina   p60 #004008  iprgc_79
## iprgc_80   iprgc_80         15927  6.298     wt_scn   p60 #08519C  iprgc_80
## iprgc_81   iprgc_81         14725  8.652    wt_dlgn   p08 #980043  iprgc_81
## iprgc_82   iprgc_82         15626  9.606    wt_dlgn   p08 #980043  iprgc_82
## iprgc_83   iprgc_83         15721 13.231  wt_retina   p08 #004008  iprgc_83
## iprgc_84   iprgc_84         16483 13.312  wt_retina   p08 #004008  iprgc_84
## iprgc_85   iprgc_85         15329  6.910     ko_scn   p08 #9BCAE1  iprgc_85
## iprgc_86   iprgc_86         16243 14.273  ko_retina   p08 #74c476  iprgc_86
## iprgc_87   iprgc_87         14022  5.647    ko_dlgn   p08 #C994C7  iprgc_87
## iprgc_88   iprgc_88         15100  7.865   het_dlgn   p08 #E7298A  iprgc_88
## iprgc_89   iprgc_89         16118 11.612    het_scn   p08 #4292C6  iprgc_89
## iprgc_90   iprgc_90         16390 23.919 het_retina   p08 #238B45  iprgc_90
## iprgc_91   iprgc_91         16375 16.016  wt_retina   p08 #004008  iprgc_91
## iprgc_92   iprgc_92         15397 10.312    wt_dlgn   p08 #980043  iprgc_92
## iprgc_93   iprgc_93         15055  8.089     wt_scn   p08 #08519C  iprgc_93
## iprgc_94   iprgc_94         14133  7.104   het_dlgn   p15 #E7298A  iprgc_94
## iprgc_95   iprgc_95         15272  7.472    het_scn   p15 #4292C6  iprgc_95
## iprgc_96   iprgc_96         16380 11.278 het_retina   p15 #238B45  iprgc_96
## iprgc_97   iprgc_97         15517 12.312     wt_scn   p15 #08519C  iprgc_97
## iprgc_98   iprgc_98         15860 14.054    wt_dlgn   p15 #980043  iprgc_98
## iprgc_99   iprgc_99         16828 17.662  wt_retina   p15 #004008  iprgc_99
## iprgc_100 iprgc_100         15386 12.256    ko_dlgn   p15 #C994C7 iprgc_100
## iprgc_101 iprgc_101         16558 12.623  ko_retina   p15 #74c476 iprgc_101
## iprgc_102 iprgc_102         15609 18.809     ko_scn   p15 #9BCAE1 iprgc_102
## iprgc_104 iprgc_104         15397 11.376   het_dlgn   p08 #E7298A iprgc_104
## iprgc_105 iprgc_105         15792 16.846   het_dlgn   p08 #E7298A iprgc_105
## iprgc_106 iprgc_106         15555 17.994   het_dlgn   p15 #E7298A iprgc_106
## iprgc_107 iprgc_107         14633  8.232    ko_dlgn   p08 #C994C7 iprgc_107
## iprgc_108 iprgc_108         15630 18.553    ko_dlgn   p15 #C994C7 iprgc_108
## iprgc_109 iprgc_109         16725 24.270  wt_retina   p08 #004008 iprgc_109
## iprgc_110 iprgc_110         15565 13.509    het_scn   p08 #4292C6 iprgc_110
## iprgc_111 iprgc_111         15703 19.606    het_scn   p08 #4292C6 iprgc_111
## iprgc_112 iprgc_112         16150 20.912    het_scn   p15 #4292C6 iprgc_112
## iprgc_113 iprgc_113         15962 12.218     ko_scn   p08 #9BCAE1 iprgc_113
## iprgc_114 iprgc_114         16311 19.403     ko_scn   p15 #9BCAE1 iprgc_114
## iprgc_115 iprgc_115         16438 19.561  wt_retina   p08 #004008 iprgc_115
## iprgc_116 iprgc_116         16926 20.965  wt_retina   p15 #004008 iprgc_116
## iprgc_117 iprgc_117         16259 18.277 het_retina   p08 #238B45 iprgc_117
## iprgc_118 iprgc_118         16500 20.885 het_retina   p08 #238B45 iprgc_118
## iprgc_119 iprgc_119         16548 16.775 het_retina   p15 #238B45 iprgc_119
## iprgc_120 iprgc_120         16561 24.538  ko_retina   p08 #74c476 iprgc_120
## iprgc_121 iprgc_121         16365 19.040  ko_retina   p08 #74c476 iprgc_121
## iprgc_122 iprgc_122         16791 21.809  ko_retina   p15 #74c476 iprgc_122
## iprgc_123 iprgc_123         14498  7.016    ko_dlgn   p08 #C994C7 iprgc_123
## iprgc_124 iprgc_124         15402  8.986     ko_scn   p08 #9BCAE1 iprgc_124
## iprgc_125 iprgc_125         15937 19.244    wt_dlgn   p08 #980043 iprgc_125
## iprgc_126 iprgc_126         15119 18.682    wt_dlgn   p08 #980043 iprgc_126
## iprgc_127 iprgc_127         16247 20.185    wt_dlgn   p15 #980043 iprgc_127
## iprgc_128 iprgc_128         15502 18.173     wt_scn   p08 #08519C iprgc_128
## iprgc_129 iprgc_129         15336 17.955     wt_scn   p08 #08519C iprgc_129
## iprgc_130 iprgc_130         16055 23.058     wt_scn   p15 #08519C iprgc_130
## 
## attr(,"class")
## [1] "hpgltools::plot_nonzero"

pp(file = "images/nonzero_v2_unfiltered.pdf")

## Warning in pp(file = "images/nonzero_v2_unfiltered.pdf"): The directory: images
## does not exist, will attempt to create it.

v2_nonzero[["plot"]]
plotted <- dev.off()

v3_nonzero  <- plot_nonzero(mm38_hisat_v3, y_intercept = 0.65)

## The following samples have less than 16526.25 genes.

##  [1] "iprgc_62"  "iprgc_63"  "iprgc_64"  "iprgc_66"  "iprgc_67"  "iprgc_68" 
##  [7] "iprgc_70"  "iprgc_71"  "iprgc_72"  "iprgc_73"  "iprgc_74"  "iprgc_75" 
## [13] "iprgc_77"  "iprgc_78"  "iprgc_80"  "iprgc_81"  "iprgc_82"  "iprgc_83" 
## [19] "iprgc_84"  "iprgc_85"  "iprgc_86"  "iprgc_87"  "iprgc_88"  "iprgc_89" 
## [25] "iprgc_90"  "iprgc_91"  "iprgc_92"  "iprgc_93"  "iprgc_94"  "iprgc_95" 
## [31] "iprgc_96"  "iprgc_97"  "iprgc_98"  "iprgc_100" "iprgc_102" "iprgc_104"
## [37] "iprgc_105" "iprgc_106" "iprgc_107" "iprgc_108" "iprgc_110" "iprgc_111"
## [43] "iprgc_112" "iprgc_113" "iprgc_114" "iprgc_115" "iprgc_117" "iprgc_118"
## [49] "iprgc_119" "iprgc_121" "iprgc_123" "iprgc_124" "iprgc_125" "iprgc_126"
## [55] "iprgc_127" "iprgc_128" "iprgc_129" "iprgc_130"

## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.

v3_nonzero

## $plot

## 
## $table
##                  id nonzero_genes    cpm  condition batch   color     label
## iprgc_62   iprgc_62         14724  2.471   het_dlgn   p15 #E7298A  iprgc_62
## iprgc_63   iprgc_63         13868  1.606   het_dlgn   p15 #E7298A  iprgc_63
## iprgc_64   iprgc_64         16488  6.404 het_retina   p15 #238B45  iprgc_64
## iprgc_65   iprgc_65         16937  7.519 het_retina   p15 #238B45  iprgc_65
## iprgc_66   iprgc_66         15455  3.599    het_scn   p15 #4292C6  iprgc_66
## iprgc_67   iprgc_67         15454  2.187    het_scn   p15 #4292C6  iprgc_67
## iprgc_68   iprgc_68         14267  2.069    ko_dlgn   p15 #C994C7  iprgc_68
## iprgc_69   iprgc_69         17055  8.448  ko_retina   p15 #74c476  iprgc_69
## iprgc_70   iprgc_70         14914  1.346     ko_scn   p15 #9BCAE1  iprgc_70
## iprgc_71   iprgc_71         15001  2.164    wt_dlgn   p15 #980043  iprgc_71
## iprgc_72   iprgc_72         15379  2.972    wt_dlgn   p15 #980043  iprgc_72
## iprgc_73   iprgc_73         15257  2.867    wt_dlgn   p15 #980043  iprgc_73
## iprgc_74   iprgc_74         16152  6.090  wt_retina   p15 #004008  iprgc_74
## iprgc_75   iprgc_75         16473  6.753  wt_retina   p15 #004008  iprgc_75
## iprgc_76   iprgc_76         17101 10.322  wt_retina   p15 #004008  iprgc_76
## iprgc_77   iprgc_77         15259  1.724     wt_scn   p15 #08519C  iprgc_77
## iprgc_78   iprgc_78         15347  3.519    wt_dlgn   p60 #980043  iprgc_78
## iprgc_79   iprgc_79         16832  7.136  wt_retina   p60 #004008  iprgc_79
## iprgc_80   iprgc_80         16030  2.680     wt_scn   p60 #08519C  iprgc_80
## iprgc_81   iprgc_81         14816  1.264    wt_dlgn   p08 #980043  iprgc_81
## iprgc_82   iprgc_82         15668  2.436    wt_dlgn   p08 #980043  iprgc_82
## iprgc_83   iprgc_83         15693  2.835  wt_retina   p08 #004008  iprgc_83
## iprgc_84   iprgc_84         16500  6.297  wt_retina   p08 #004008  iprgc_84
## iprgc_85   iprgc_85         15361  1.757     ko_scn   p08 #9BCAE1  iprgc_85
## iprgc_86   iprgc_86         16220  6.511  ko_retina   p08 #74c476  iprgc_86
## iprgc_87   iprgc_87         14135  1.844    ko_dlgn   p08 #C994C7  iprgc_87
## iprgc_88   iprgc_88         15167  2.582   het_dlgn   p08 #E7298A  iprgc_88
## iprgc_89   iprgc_89         16164  4.600    het_scn   p08 #4292C6  iprgc_89
## iprgc_90   iprgc_90         16391 10.355 het_retina   p08 #238B45  iprgc_90
## iprgc_91   iprgc_91         16326  7.632  wt_retina   p08 #004008  iprgc_91
## iprgc_92   iprgc_92         15414  2.533    wt_dlgn   p08 #980043  iprgc_92
## iprgc_93   iprgc_93         15091  1.392     wt_scn   p08 #08519C  iprgc_93
## iprgc_94   iprgc_94         14278  2.504   het_dlgn   p15 #E7298A  iprgc_94
## iprgc_95   iprgc_95         15341  2.114    het_scn   p15 #4292C6  iprgc_95
## iprgc_96   iprgc_96         16360  5.415 het_retina   p15 #238B45  iprgc_96
## iprgc_97   iprgc_97         15561  2.966     wt_scn   p15 #08519C  iprgc_97
## iprgc_98   iprgc_98         15954  6.228    wt_dlgn   p15 #980043  iprgc_98
## iprgc_99   iprgc_99         16813  8.527  wt_retina   p15 #004008  iprgc_99
## iprgc_100 iprgc_100         15437  4.731    ko_dlgn   p15 #C994C7 iprgc_100
## iprgc_101 iprgc_101         16637  6.179  ko_retina   p15 #74c476 iprgc_101
## iprgc_102 iprgc_102         15646  3.554     ko_scn   p15 #9BCAE1 iprgc_102
## iprgc_104 iprgc_104         15445  2.058   het_dlgn   p08 #E7298A iprgc_104
## iprgc_105 iprgc_105         15820  3.737   het_dlgn   p08 #E7298A iprgc_105
## iprgc_106 iprgc_106         15741  6.928   het_dlgn   p15 #E7298A iprgc_106
## iprgc_107 iprgc_107         14751  2.603    ko_dlgn   p08 #C994C7 iprgc_107
## iprgc_108 iprgc_108         15704  7.611    ko_dlgn   p15 #C994C7 iprgc_108
## iprgc_109 iprgc_109         16726 10.979  wt_retina   p08 #004008 iprgc_109
## iprgc_110 iprgc_110         15609  2.306    het_scn   p08 #4292C6 iprgc_110
## iprgc_111 iprgc_111         15700  2.897    het_scn   p08 #4292C6 iprgc_111
## iprgc_112 iprgc_112         16170  5.384    het_scn   p15 #4292C6 iprgc_112
## iprgc_113 iprgc_113         16015  2.656     ko_scn   p08 #9BCAE1 iprgc_113
## iprgc_114 iprgc_114         16350  7.459     ko_scn   p15 #9BCAE1 iprgc_114
## iprgc_115 iprgc_115         16401  8.515  wt_retina   p08 #004008 iprgc_115
## iprgc_116 iprgc_116         16897  9.145  wt_retina   p15 #004008 iprgc_116
## iprgc_117 iprgc_117         16241  8.451 het_retina   p08 #238B45 iprgc_117
## iprgc_118 iprgc_118         16472  9.422 het_retina   p08 #238B45 iprgc_118
## iprgc_119 iprgc_119         16519  7.738 het_retina   p15 #238B45 iprgc_119
## iprgc_120 iprgc_120         16554 10.172  ko_retina   p08 #74c476 iprgc_120
## iprgc_121 iprgc_121         16391  8.346  ko_retina   p08 #74c476 iprgc_121
## iprgc_122 iprgc_122         16746  8.757  ko_retina   p15 #74c476 iprgc_122
## iprgc_123 iprgc_123         14654  2.195    ko_dlgn   p08 #C994C7 iprgc_123
## iprgc_124 iprgc_124         15499  2.064     ko_scn   p08 #9BCAE1 iprgc_124
## iprgc_125 iprgc_125         15973  3.665    wt_dlgn   p08 #980043 iprgc_125
## iprgc_126 iprgc_126         15131  1.888    wt_dlgn   p08 #980043 iprgc_126
## iprgc_127 iprgc_127         16232  5.944    wt_dlgn   p15 #980043 iprgc_127
## iprgc_128 iprgc_128         15487  2.125     wt_scn   p08 #08519C iprgc_128
## iprgc_129 iprgc_129         15306  1.645     wt_scn   p08 #08519C iprgc_129
## iprgc_130 iprgc_130         16041  5.784     wt_scn   p15 #08519C iprgc_130
## 
## attr(,"class")
## [1] "hpgltools::plot_nonzero"

pp(file = "images/nonzero_v3_unfiltered.pdf")
v3_nonzero[["plot"]]
plotted <- dev.off()

Oh wow, I did not expect such a profound effect on the cpm values on the more saturated libraries. I guess in retrospect I should have?

Also note to self, we are not messing with p60.

7.1 Exclude p60

mm38_hisat_v2 <- subset_se(mm38_hisat_v2, subset = "time_atb!='p60'")
mm38_hisat_v3 <- subset_se(mm38_hisat_v3, subset = "time_atb!='p60'")

7.2 Replot the nonzero gene plots

v2_nonzero_filt <- plot_nonzero(mm38_hisat_v2, plot_labels = FALSE)

## The following samples have less than 16526.25 genes.

##  [1] "iprgc_62"  "iprgc_63"  "iprgc_64"  "iprgc_66"  "iprgc_67"  "iprgc_68" 
##  [7] "iprgc_70"  "iprgc_71"  "iprgc_72"  "iprgc_73"  "iprgc_74"  "iprgc_75" 
## [13] "iprgc_77"  "iprgc_81"  "iprgc_82"  "iprgc_83"  "iprgc_84"  "iprgc_85" 
## [19] "iprgc_86"  "iprgc_87"  "iprgc_88"  "iprgc_89"  "iprgc_90"  "iprgc_91" 
## [25] "iprgc_92"  "iprgc_93"  "iprgc_94"  "iprgc_95"  "iprgc_96"  "iprgc_97" 
## [31] "iprgc_98"  "iprgc_100" "iprgc_102" "iprgc_104" "iprgc_105" "iprgc_106"
## [37] "iprgc_107" "iprgc_108" "iprgc_110" "iprgc_111" "iprgc_112" "iprgc_113"
## [43] "iprgc_114" "iprgc_115" "iprgc_117" "iprgc_118" "iprgc_121" "iprgc_123"
## [49] "iprgc_124" "iprgc_125" "iprgc_126" "iprgc_127" "iprgc_128" "iprgc_129"
## [55] "iprgc_130"

## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.

## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.
## Not putting labels on the plot.

pp(file = "images/nonzero_v2_filt.pdf")
v2_nonzero_filt[["plot"]]
plotted <- dev.off()

v3_nonzero_filt <- plot_nonzero(mm38_hisat_v3, plot_labels = FALSE)

## The following samples have less than 16526.25 genes.

##  [1] "iprgc_62"  "iprgc_63"  "iprgc_64"  "iprgc_66"  "iprgc_67"  "iprgc_68" 
##  [7] "iprgc_70"  "iprgc_71"  "iprgc_72"  "iprgc_73"  "iprgc_74"  "iprgc_75" 
## [13] "iprgc_77"  "iprgc_81"  "iprgc_82"  "iprgc_83"  "iprgc_84"  "iprgc_85" 
## [19] "iprgc_86"  "iprgc_87"  "iprgc_88"  "iprgc_89"  "iprgc_90"  "iprgc_91" 
## [25] "iprgc_92"  "iprgc_93"  "iprgc_94"  "iprgc_95"  "iprgc_96"  "iprgc_97" 
## [31] "iprgc_98"  "iprgc_100" "iprgc_102" "iprgc_104" "iprgc_105" "iprgc_106"
## [37] "iprgc_107" "iprgc_108" "iprgc_110" "iprgc_111" "iprgc_112" "iprgc_113"
## [43] "iprgc_114" "iprgc_115" "iprgc_117" "iprgc_118" "iprgc_119" "iprgc_121"
## [49] "iprgc_123" "iprgc_124" "iprgc_125" "iprgc_126" "iprgc_127" "iprgc_128"
## [55] "iprgc_129" "iprgc_130"

## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.
## Not putting labels on the plot.

pp(file = "images/nonzero_v3_filt.pdf")
v3_nonzero_filt[["plot"]]
plotted <- dev.off()

Once again, I do not want to lose the previous code, so here is the v1 invocation

mm38_hisat_v1 <- subset_se(mm38_hisat_v1, subset = "time_atb!='p60'")

8 Quick PCA, then return to Theresa’s document

v2_norm <- normalize(mm38_hisat_v2, transform = "log2", convert = "cpm",
                     norm = "quant", filter = TRUE)

## Removing 10298 low-count genes (15127 remaining).

## transform_counts: Found 8465 values equal to 0, adding 1 to the matrix.

v2_norm_pca <- plot_pca(v2_norm)
v2_norm_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by het_dlgn, het_retina, het_scn, ko_dlgn, ko_retina, ko_scn, wt_dlgn, wt_retina, wt_scn
## Shapes are defined by p08, p15.

pp(file = "images/v2_norm_pca.pdf")
v2_norm_pca[["plot"]]
plotted <- dev.off()

v3_norm <- normalize(mm38_hisat_v3, transform = "log2", convert = "cpm",
                     norm = "quant", filter = TRUE)

## Removing 10156 low-count genes (15269 remaining).

## transform_counts: Found 9347 values equal to 0, adding 1 to the matrix.

v3_norm_pca <- plot_pca(v3_norm)
v3_norm_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by het_dlgn, het_retina, het_scn, ko_dlgn, ko_retina, ko_scn, wt_dlgn, wt_retina, wt_scn
## Shapes are defined by p08, p15.

pp(file = "images/v3_norm_pca.pdf")
v3_norm_pca[["plot"]]
plotted <- dev.off()

Ibid.

v1_norm <- normalize(mm38_hisat_v1, transform = "log2", convert = "cpm",
                     norm = "quant", filter = TRUE)
plot_pca(v1_norm)

To my eyes it looks like we just have 1 weirdo p15 sample? Deduplication had a minor but significant effect on the PCA.

With that in mind, let us look at Theresa’s WORKING document and see what we can recapitulate.

Theresa’s document: The TRAP protocol has some variability which is introduced at different stpdf including homogenization, antibody labeling, pulldown efficiency/specificity, sample handling during cleanup stpdf, and library prep/sequencing. We know from Rashmi’s QC that there is variability at the level of pulldown efficiency (amount of RNA isolated). She is doing a good job of keeping track of this for all her samples and we have validated her P8 results (attached supplementary figure 3D). We consistently see clear differences between control and cre samples for the retina, which makes sense because the cell bodies are in the retina. The target tissue differences are smaller, which also makes sense for axon-TRAP. We think that some of her P15 samples are not good based on low amounts of isolated RNA from cre(+) retina samples. We plan to drop these samples and not perform additional isolations at this time point. Based on this (and the general lack of large developmental effects), we were planning to focus on presenting the P8 data only in the paper. Interested to hear your thoughts in this…

My notes: Theresa’s first operations in this notebook were to:

1. Set location as condition, genotype as batch.
2. Perform PCA before/after sva.

v3_loc_geno <- set_conditions(mm38_hisat_v3, fact = "location_atb",
                              colors = color_choices[["location"]]) %>%
  set_batches(fact = "genotype_atb")

## The numbers of samples by condition are:

## 
##   dlgn retina    scn 
##     23     23     19

## The number of samples by batch are:

## 
## het  ko  wt 
##  21  18  26

8.1 The associated PCA

At different times, it appears to me that Theresa has preferred slightly different normalization methods, primarily a mix of TMM and quantile.

Thus I will use different suffix letters to denote various normalizations employed, and if they turn out the same I will pick one arbitrarily.

loc_geno_nq <- normalize(v3_loc_geno, transform = "log2", convert = "cpm",
                         filter = TRUE, norm = "quant")

## Removing 10156 low-count genes (15269 remaining).

## transform_counts: Found 9347 values equal to 0, adding 1 to the matrix.

location_genotype_pca <- plot_pca(loc_geno_nq)
pp(file = "images/location_genotype_norm_pca.pdf")
location_genotype_pca[["plot"]]
plotted <- dev.off()
location_genotype_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by dlgn, retina, scn
## Shapes are defined by het, ko, wt.

## ok, I have two weirdo samples which look very much like they are actually dlgn.
## These are sample IDs iprgc_66 and iprgc_130

loc_geno_nt <- normalize(v3_loc_geno, transform = "log2", convert = "cpm",
                         filter = TRUE, norm = "tmm")

## Removing 10156 low-count genes (15269 remaining).

## transform_counts: Found 42869 values equal to 0, adding 1 to the matrix.

location_genotype_tmm_pca <- plot_pca(loc_geno_nt)
pp(file = "images/location_genotype_tmm_pca.pdf")
location_genotype_tmm_pca[["plot"]]

## Warning in MASS::cov.trob(data[, vars], wt = weight * nrow(data)): Probable
## convergence failure
## Warning in MASS::cov.trob(data[, vars], wt = weight * nrow(data)): Probable
## convergence failure

plotted <- dev.off()
location_genotype_tmm_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by dlgn, retina, scn
## Shapes are defined by het, ko, wt.

## Warning in MASS::cov.trob(data[, vars], wt = weight * nrow(data)): Probable
## convergence failure
## Warning in MASS::cov.trob(data[, vars], wt = weight * nrow(data)): Probable
## convergence failure

A random thought about these PCA plots, it might be worth while to add a panel below the legend with the sample numbers per condition/batch.

Of course, the same information is provided in a more fun fashion via my silly sankey function:

sample_sankey <- plot_meta_sankey(v3_loc_geno, color_choices = color_choices,
                                  factors = c("genotype_atb", "location_atb", "time_atb"))

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Warning: The `size` argument of `element_rect()` is deprecated as of ggplot2 3.4.0.
## i Please use the `linewidth` argument instead.
## i The deprecated feature was likely used in the ggsankey package.
##   Please report the issue at <https://github.com/davidsjoberg/ggsankey/issues>.
## This warning is displayed once per session.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

pp(file = "images/design_sankey.pdf")
sample_sankey[["ggplot"]]
plotted <- dev.off()
sample_sankey

## A sankey plot describing the metadata of 65 samples,
## including 30 out of 0 nodes and traversing metadata factors:
## genotype_atb, location_atb, time_atb.

9 A Short conversation with Rashmi

Rashmi came by and we discussed the samples a little. She suggested that is likely that we will need to exclude the 202205 samples, these may be identified by a few ways, most easily I think via the ‘project_ah’ column, they are the 021_1 samples.

My sense was that she concurred with my interpretation of the umi deduplication, so I will continue using the deduplicated results exclusively, at least for now.

10 Melanopsin Sanity Check

One of Theresa’s first checks was wisely for melanopsin. Let us repeat a version of this:

opn4_exprs <- data.frame(combined = colData(loc_geno_nt)[["geno_loc_atb"]],
                         location = colData(loc_geno_nt)[["location_atb"]],
                         genotype = colData(loc_geno_nt)[["genotype_atb"]],
                         opn = exprs(loc_geno_nt)["ENSMUSG00000021799", ])

groupedstats::grouped_summary(opn4_exprs, location, opn)

## Error in `loadNamespace()`:
## ! there is no package called 'groupedstats'

opn4_location <- ggbetweenstats(data = opn4_exprs, x = location, y = opn)
pp(file = "images/ggbetween_location.pdf")
opn4_location
plotted <- dev.off()
opn4_location

opn4_genotype <- ggbetweenstats(data = opn4_exprs, x = genotype, y = opn)
pp(file = "images/ggbetween_location.pdf")
opn4_genotype
plotted <- dev.off()
opn4_genotype

opn4_combined <- ggbetweenstats(data = opn4_exprs, x = combined, y = opn)

## Warning: x Number of labels is greater than default palette color count.
## i Select another color `palette` (and/or `package`).

pp(file = "images/ggbetween_combined.pdf")
opn4_combined
plotted <- dev.off()
opn4_combined

ok, so I plotted the question a bit differently, but got the same answer.

Here is the text of Theresa’s notebook following this analysis:

“Ugh oh, looks like there is at least one retina KO sample that has some melanopsin expression in it. Turns out ipRGC_07 is a bad egg which is supposed to be a KO but has melanopsin expression. It’s friends which were pooled from the same mice are iprgc_06 and iprgc_08, so we need to exclude all these samples.”

I am also seeing some knockout expression with some caveats: I do not have the affected samples in my dataset (iprgc_07) and the levels I am seeing are quite low – I will look in IGV to double check, but I strongly suspect that these are some piddly reads near the UTRs.

Onward!

11 PCA plots

11.1 PCA of all genes by location

Theresa’s first pca was of log2 cpm values. I might add quantile/tmm to this?

v3_location <- set_conditions(mm38_hisat_v3, fact = "location_atb") %>%
  set_batches(fact = "genotype_atb") %>%
  set_colors(color_choices[["location"]])

## The numbers of samples by condition are:

## 
##   dlgn retina    scn 
##     23     23     19

## The number of samples by batch are:

## 
## het  ko  wt 
##  21  18  26

v3_location_norm <- normalize(v3_location, filter = TRUE, norm = "quant",
                                   transform = "log2", convert = "cpm")

## Removing 10156 low-count genes (15269 remaining).

## transform_counts: Found 9347 values equal to 0, adding 1 to the matrix.

plot_pca(v3_location_norm)

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by dlgn, retina, scn
## Shapes are defined by het, ko, wt.

Once again we see that samples iprgc_66 and iprgc_130 are likely actually DLGN and not SCN. I am therefore going to add a column to the sample sheet noting this, and remove them from the expressionset.

I will thus replot the data after removing those two. If we want to see what it looks like with the re-attributed locations, we can do so.

Theresa has a nice change to the PCA plotter in which she sets the alpha channel as an additional visual queue for a metadata factor…

mm38_hisat_v3 <- subset_se(mm38_hisat_v3, subset="sampleid!='iprgc_130'") %>%
  subset_se(subset="sampleid!='iprgc_66'")
v3_location <- set_conditions(mm38_hisat_v3, fact = "location_atb") %>%
  set_batches(fact = "genotype_atb") %>%
  set_colors(color_choices[["location"]])

## The numbers of samples by condition are:

## 
##   dlgn retina    scn 
##     23     23     17

## The number of samples by batch are:

## 
## het  ko  wt 
##  20  18  25

v3_location_norm <- normalize(v3_location, filter = TRUE, norm = "quant",
                                   transform = "log2", convert = "cpm")

## Removing 10162 low-count genes (15263 remaining).

## transform_counts: Found 8867 values equal to 0, adding 1 to the matrix.

filtered_location_pca <- plot_pca(v3_location_norm)
pp(file = "images/filtered_location_pca.pdf")
filtered_location_pca[["plot"]]
plotted <- dev.off()
filtered_location_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by dlgn, retina, scn
## Shapes are defined by het, ko, wt.

removed_sankey <- plot_meta_sankey(v3_location, color_choices = color_choices,
                                   factors = c("genotype_atb", "location_atb", "time_atb"))

## Warning: attributes are not identical across measure variables; they will be
## dropped

pp(file = "images/filtered_sankey.pdf")
removed_sankey[["ggplot"]]
plotted <- dev.off()
removed_sankey

## A sankey plot describing the metadata of 63 samples,
## including 30 out of 0 nodes and traversing metadata factors:
## genotype_atb, location_atb, time_atb.

Here is Theresa’s text, recall once again that I do not have some of these older samples (iprgc_62):

PC1 vs PC2 identifies retina vs axon is still the main component of variation. We do see though that in the PC2 direction, we see with the new samples added, we don’t see separation based on axonal targets (dLGN vs SCN). In the PC1 vs PC3 plot, we see that it’s PC3 where we start to see variation correlated with axonal compartment. Let’s look at PC1 vs PC2 colored by batch (when they were processed/sequenced) to see if that is what is contributing so much variation in PC2.

Side note: ipRGC 62 seems like an odd ball. This seems to me like it should have been a dLGN P08 sample. Is there any possibility this got mislabeled early on? I went back and double checked to see if all my processing is correct and it indeed was labeled an SCN P15 from the time I got the samples, and it is indeed.

12 DE

I now switched to Theresa’s document ‘WORKING_axonTRAP…’ and will start pulling sections from it. I am reasonably certain I have reasonably similar sample distributions, so I presume I can invoke similar/identical calls for DESeq and friends.

12.1 p8 retinas

In the block immediately before the DE analyses, Theresa created a subset expressionset of only p08 retinas. Thus this initial DE I assume will be used to subtract for the SCN/DLGN analyses that follow. (I guess I could read ahead and find out, but no! I want to be a blank slate)

Theresa’s primary workflow makes heavy use of DESeq2 (Love, Huber, and Anders (2014)) and sva (Leek et al. (2012)). In some(most?) of Theresa’s invocations of the all_pairwise() function, she excludes the other methods that it performs. In this workbook, I left those methods on, thus we can evaluate the relative performance DESeq2 vs. some (all? I may have disabled EBSeq/dream because they were taking too long) of the following:

• limma: (Ritchie et al. (2015)) (among other references) originally written for microarrays.
• EdgeR: (Robinson, McCarthy, and Smyth (2010)), which shares many assumptions with DESeq2.
• EBSeq: (Leng et al. (2013)), because I have a soft spot for any Bayesian method.
• Noiseq: (Tarazona et al. (2011)), which seeks to directly model variance in an RNASeq dataset and use that to improve the sensitivity of the result, much like:
• Dream: (Gabriel E. Hoffman and Roussos (2020)), written by the same authors (and uses very similar logic) as one of my favorite tools, variancePartition(Gabriel E. Hoffman and Schadt (2016)).

mm38_p8_retina <- subset_se(mm38_hisat_v3, subset = "time_atb=='p08' & location_atb=='retina'")
mm_normal_p8_ret_de <- all_pairwise(mm38_p8_retina, model_svs = "svaseq",
                                    model_fstring = "~ 0 + condition", filter = TRUE)

## het_retina  ko_retina  wt_retina 
##          3          3          5

## Removing 12001 low-count genes (13424 remaining).

## Basic step 0/3: Normalizing data.

## Basic step 0/3: Converting data.

## I think this is failing? SummarizedExperiment

## Basic step 0/3: Transforming data.

## Setting 2593 entries to zero.

## This received a matrix of SVs.

## converting counts to integer mode

## gene-wise dispersion estimates

## mean-dispersion relationship

## final dispersion estimates

## conditions
## het_retina  ko_retina  wt_retina 
##          3          3          5

## conditions
## het_retina  ko_retina  wt_retina 
##          3          3          5

## conditions
## het_retina  ko_retina  wt_retina 
##          3          3          5

mm_normal_p8_ret_de

## A pairwise differential expression with results from: basic, deseq, ebseq, edger, limma, noiseq.

## This used a surrogate/batch estimate from: svaseq.

## The primary analysis performed 3 comparisons.

The following invocation performed by Theresa filters the wt/het comparison for only those genes which increased by at least 0.25 logFC with a significant adjusted p-value. I assume that this is to use the wt samples as a translational control for the ket/ko comparisons; I am therefore thinking that for my purposes, I will therefore separate the contrasts from all_pairwise do this in a stepwise fashion…

The block of code immediately following Theresa’s all_pairwise() invocation is a little confusing for me and warrants some explanation by me to me in the hopes that I do not misunderstand what is happening and the goals therein.

I think I can safely assume that the goal here is to pull out the IDs which increased in het with respect to wild type; even if by a small margin, as long as it is statistically significant vis a vis the adjusted p-value.

I am going to perform what I think is the same thing in a slightly different fashion so that I can share a copy of the results with whomever is interested. I will also repeat Theresa’s invocation and prove to myself that I understood and got the same answer.

wt_het_keeper <- list("het_vs_wt" = c("het_retina", "wt_retina"))
het_wt_table <- combine_de_tables(mm_normal_p8_ret_de, keepers = wt_het_keeper,
                                  label_column = label_column,
                                  excel = "excel/het_retina_control.xlsx")

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

wanted_sig <- extract_significant_genes(het_wt_table,
                                        lfc = 0.25,
                                        according_to = "deseq")
wanted_het_increased <- wanted_sig[["deseq"]][["ups"]][["het_vs_wt"]]
increased_het_genes <- rownames(wanted_het_increased)

Here are Theresa’s next lines:

mm_de_normal_p8_ret <- mm_normal_p8_ret_de
hetkeeper_genes <- mm_de_normal_p8_ret$deseq$all_tables$wt_retina_vs_het_retina %>%
  filter(logFC <= -0.25 & adj.P.Val <= 0.05)
kokeeper_genes <- mm_de_normal_p8_ret$deseq$all_tables$wt_retina_vs_ko_retina %>%
  filter(logFC <= -0.25 & adj.P.Val <= 0.05)
keepergenes <- unique(c(rownames(hetkeeper_genes),
                        rownames(kokeeper_genes)))
## We know a priori that Opn4 is ENSMUSG00000021799
## I do not expect to see it in this set, it should be higher in wt
## retina vs ko retina by a significant margin.
"ENSMUSG00000021799" %in% keepergenes

## [1] TRUE

## Oooohhh but it _is_ higher in het vs. wt, as we saw in
## the violin plot earlier.

I think Rashmi made a compelling point which illustrates why we likely should expect the expression of Opn4 to significantly higher in the heterozygotes vs wild-type:

1. Recall that the assay is using the immunopurification to extract the RNAs.
2. The wt samples do not have the cre recombinase and therefore no HA and therefore everything we observe is due to non-specific binding.
3. The set of genes observed due to non-specific binding is different than het/ko (presumably a larger number of relatively small values), therefore the divisor performed in the cpm is likely relativly large resulting in normalized values getting shifted down to some degree.
4. On the other hand, the set of genes observed in het/ko are more likely to be only the specific binders and therefore smaller (I can test this) resulting in a smaller divisor and slight shifting up in the cpm values.

This makes me wonder if any normalization methods exist which do something like multiply the values by some value related to the proportion of observed genes; and/or if this is a good/bad/indifferent idea.

Also, just a note for me to remember: RPL22, not RPS22, for some reason I keep thinking the small subunit.

12.2 Prove I understood

hetkeeper_genes <- mm_normal_p8_ret_de$deseq$all_tables$wt_retina_vs_het_retina %>%
  filter(logFC <= -0.25 & adj.P.Val <= 0.05)
testthat::expect_true(nrow(hetkeeper_genes) == length(increased_het_genes))
taa_keepers <- sort(rownames(hetkeeper_genes))
atb_keepers <- sort(increased_het_genes)
testthat::expect_equal(taa_keepers, atb_keepers)

Yay! I can read! Now let us repeat for the KO vs wt

wt_ko_keeper <- list("ko_vs_wt" = c("ko_retina", "wt_retina"))
ko_wt_table <- combine_de_tables(mm_normal_p8_ret_de, keepers = wt_ko_keeper,
                                 label_column = label_column,
                                 excel = "excel/ko_retina_control.xlsx")

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

wanted_sig <- extract_significant_genes(ko_wt_table,
                                        lfc = 0.25,
                                        according_to = "deseq")
wanted_ko_increased <- wanted_sig[["deseq"]][["ups"]][["ko_vs_wt"]]
increased_ko_genes <- rownames(wanted_ko_increased)

The next thing performed in Theresa’s document is a unique(concatenation of these two gene groups), thus sucking up every gene which was significantly higher in either the knockout or heterzyous samples with respect to wild-type.

This was followed by a couple of merge operations of a little bit of the annotation data; I am not sure I understand the goal yet…

Here is her code. I copied the annotation ‘mgi_symbol’ column to ‘external_gene_name’ so that I need not change any of her code. I am assuming this is the appropriate column of interest, I do not know this for certain, but it seems quite likely.

While I am at it, here is the set_sig_limma() function from Theresa’s helpers.R

set_sig_limma <- function(limma_tbl, factors = NULL) {
  if (is.null(factors)) {
    #set significance for plotting colors
    limma_tbl$Significance <- NA
    limma_tbl[abs(limma_tbl$logFC) < 1 | limma_tbl$adj.P.Val > .05, "Significance"] <- "Not \nEnriched"
    limma_tbl[limma_tbl$logFC >= 1  & limma_tbl$adj.P.Val <= .05, ][["Significance"]] <- "Disease \nUpregulated"
    limma_tbl[limma_tbl$logFC <= -1  & limma_tbl$adj.P.Val <= .05, ][["Significance"]] <- "Disease \nDownregulated"
    limma_tbl$Significance <- factor(limma_tbl$Significance, levels = c("Upregulated", "Downregulated",  "Not \nEnriched"))
  } else {
    limma_tbl$Significance <- NA
    limma_tbl[abs(limma_tbl$logFC) < 1 | limma_tbl$adj.P.Val > .05, "Significance"] <- "Not \nEnriched"
    if(nrow(limma_tbl[limma_tbl$logFC >= 1  & limma_tbl$adj.P.Val <= .05, ]) != 0) {
      limma_tbl[limma_tbl$logFC >= 1  & limma_tbl$adj.P.Val <= .05, ][["Significance"]] <- factors[1]
    }
    if (nrow(limma_tbl[limma_tbl$logFC <= -1  & limma_tbl$adj.P.Val <= .05, ]) != 0) {
      limma_tbl[limma_tbl$logFC <= -1  & limma_tbl$adj.P.Val <= .05, ][["Significance"]] <- factors[2]
    }
    limma_tbl$Significance <- factor(limma_tbl$Significance, levels = c(factors,  "Not \nEnriched"))
  }
  return(limma_tbl)
}

12.2.1 Combining het/wt and ko/wt

mm_annot[["external_gene_name"]] <- mm_annot[["mgi_symbol"]]
keepergenes <- unique(c(rownames(hetkeeper_genes), rownames(kokeeper_genes)))
length(keepergenes)

## [1] 3632

annots_to_merge <- mm_annot %>%
  select(ensembl_gene_id, external_gene_name) %>%
  filter(ensembl_gene_id %in%
           rownames(mm_de_normal_p8_ret$deseq$all_tables$ko_retina_vs_het_retina)) %>%
  distinct()

## Error in `select()` at dplyr/R/filter.R:219:3:
## ! Can't select columns that don't exist.
## x Column `external_gene_name` doesn't exist.

mm_de_normal_p8_ret$deseq$all_tables$ko_retina_vs_het_retina <- merge(
  mm_de_normal_p8_ret$deseq$all_tables$ko_retina_vs_het_retina, annots_to_merge,
  by.x = 0, by.y = "ensembl_gene_id", all.x = TRUE)

## Error:
## ! object 'annots_to_merge' not found

df <- mm_de_normal_p8_ret$deseq$all_tables$ko_retina_vs_het_retina %>%
  dplyr::mutate(logFC = -logFC) %>%
  set_sig_limma(factors = c("Het Enriched", "KO Enriched"))

My version of the above task makes use of the excludes option of combine_de_tabes. Given the set of unique gene IDs increased in the het/ko, I can ask to exlude anything not in that set. I could also have more parsimoniously directly excluded any gene ID increased in the wt samples. But, Theresa already provided the code to do the former, so it will be less typing/opportunity for silly mistakes to just do that.

both_increased_genes <- unique(c(increased_het_genes, increased_ko_genes))
## arbitrairly grab all genes from one of my data structures.
all_genes <- rownames(exprs(mm38_hisat_v3))
exclude_idx <- all_genes %in% both_increased_genes
summary(exclude_idx)

##    Mode   FALSE    TRUE 
## logical   21793    3632

exclude_increased_genes <- all_genes[exclude_idx]
retina_keepers <- list(
  "het_vs_wt" = c("het_retina", "wt_retina"),
  "ko_vs_wt" = c("ko_retina", "wt_retina"),
  "ko_vs_het" = c("ko_retina", "het_retina"))
## A reminder to myself: there is also a parameter 'wanted_genes'
## which does effectively the same thing as excludes in this context;
## excludes was originally written to allow flexible, keyword-based
## exclusion.
p8_retina_tables <- combine_de_tables(
  mm_normal_p8_ret_de, keepers = retina_keepers,
  wanted_genes = both_increased_genes, label_column = label_column,
  excel = glue("excel/p8_retina_kept_genes_increased_in_wt_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

p8_retina_sig <- extract_significant_genes(
  p8_retina_tables,
  excel = glue("excel/p8_retina_kept_genes_increased_in_wt_sig-v{ver}.xlsx"),
  according_to = "deseq")
opposite_p8_retina_tables <- combine_de_tables(
  mm_normal_p8_ret_de, keepers = retina_keepers,
  excludes = both_increased_genes, label_column = label_column,
  excel = glue("excel/p8_retina_removed_genes_increased_in_wt_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.
## Looking for subscript invalid names, end of extract_keepers.

opposite_p8_retina_sig <- extract_significant_genes(
  p8_retina_tables,
  excel = glue("excel/p8_retina_removed_genes_increased_in_wt_sig-v{ver}.xlsx"),
  according_to = "deseq")

13 Filtering out non-specific genes and examining the results

The following is a copy/paste from Theresa containing the remaining tasks she performed and will provide the template for implementation of the final tasks.

This picks up with the lines from her notebook immediately following the invocation of ‘set_sig_limma(factors = c(“Het Enriched” …’.

For all of the remaining blocks I will copy in her code, turn off its evaluation, run the blocks manually, compare them to her notebook output, then enable each block as I ensure I understand it.

I will likely therefore introduce some small formatting changes and add some additional GSEA/enrichment tasks once the non-specific filtering is complete.

df <- df %>%
  filter(Row.names %in% keepergenes)

## Error in `filter()`:
## i In argument: `Row.names %in% keepergenes`.
## Caused by error:
## ! object 'Row.names' not found

labels_ups <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1) %>%
  arrange(logFC) %>%
  head(n = 9)
labels_downs <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1) %>%
  arrange(-logFC) %>%
  head(n = 11)
labels <- rbind(labels_ups, labels_downs)
res_tbl <- df
DEplot <- ggplot(res_tbl, aes(x = logFC, y = -log10(adj.P.Val), label = external_gene_name)) +
  geom_point(aes(colour = Significance), size = 4) +
  geom_vline(xintercept = c(-1, 1)) +
  geom_hline(yintercept = -log10(0.05)) +
  theme_classic(base_size = 20) +
  xlab("log2(FC)") +
  ylab("-log10(p-value)") +
  theme(legend.position = "right") +
  scale_color_manual(values = c("#F8766D", "#00BFC4", "Grey")) +
  geom_label_repel(
    data = filter(df,
                  ## c('s5_het_dlgn', 's5_het_ret', 's5_het_scn')),
                  external_gene_name %in% labels$external_gene_name),
    ## nudge_x = -0.5,
    nudge_y = 3, max.overlaps = 15) +
  xlim(c(-3, 6))

## Error in `filter()` at ggrepel/R/geom-label-repel.R:49:3:
## i In argument: `external_gene_name %in% labels$external_gene_name`.
## Caused by error:
## ! object 'external_gene_name' not found

pp(file = "images/p08_retina_DE_1312024.pdf")
DEplot

## Error:
## ! object 'DEplot' not found

plotted <- dev.off()
DEplot

## Error:
## ! object 'DEplot' not found

writexl::write_xlsx(df, path = "excel/retinahet_vs_retinako_WTfiltered.xlsx")

## Error in `loadNamespace()`:
## ! there is no package called 'writexl'

13.1 How many ups/downs

ko_enriched <- df %>%
  filter(Significance == "KO Enriched")
nrow(ko_enriched)

## [1] 88

het_enriched <-  df %>%
  filter(Significance == "Het Enriched")
nrow(het_enriched)

## [1] 137

13.2 category enrichment/GSEA

regulated_genes <- res_tbl %>%
  filter(adj.P.Val <= 0.05) %>%
  arrange(logFC) %>%
  select(Row.names, logFC, adj.P.Val, external_gene_name, Significance) %>%
  filter(abs(logFC) >= 1)

## Error in `select()` at dplyr/R/filter.R:219:3:
## ! Can't select columns that don't exist.
## x Column `Row.names` doesn't exist.

## gsea_result_ko <- gost(query = ko_genes$external_gene_name,
##                        organism = "mmusculus",
##                        evcodes = TRUE,
##                        ordered_query = TRUE)
gsea_result_het <- gost(query = het_enriched$external_gene_name,
                        organism = "mmusculus",
                        evcodes = TRUE,
                        ordered_query = TRUE)

## Missing query

##gsea_result_alldysregulated <- gost(query = alldysregulated_genes$external_gene_name,
##                                    organism = "mmusculus",
##                                    evcodes = TRUE,
##                                    ordered_query = TRUE)

I have a function in my package which seeks to make gProfiler queries a bit more complete and easy. Let us see how similar the result is…

rownames(alldysregulated_genes) <- alldysregulated_genes[["Row.names"]]
alldysregulated_genes[["Row.names"]] <- NULL

het_gp <- simple_gprofiler(rownames(alldysregulated_genes),
                           species = "mmusculus",
                           excel = glue("excel/het_gprofiler-v{ver}.xlsx"))
het_gp
enrichplot::dotplot(het_gp[["BP_enrich"]])
gp_pair <- enrichplot::pairwise_termsim(het_gp[["BP_enrich"]])
enrichplot::emapplot(gp_pair)
enrichplot::ssplot(gp_pair)
enrichplot::treeplot(gp_pair)
upsetplot(het_gp[["BP_enrich"]])

enrichplot::dotplot(het_gp[["REAC_enrich"]])
gp_pair <- enrichplot::pairwise_termsim(het_gp[["REAC_enrich"]])
enrichplot::emapplot(gp_pair)
enrichplot::ssplot(gp_pair)
enrichplot::treeplot(gp_pair)
upsetplot(het_gp[["REAC_enrich"]])

I make a somewhat arbitrary distinction between the concepts of over-enrichment analyses and GSEA: the former (as performed by gprofiler) (Raudvere et al. (2019)) seeks to find groups of genes overrepresented in GO/reactome/etc. These groups of genes are taken exclusively from the top-n/bottom-n genes with respect to fold-change between conditions of interest; in this case most different than wt in the p08 retina ko or het samples.

With that in mind, I can invoke a similar function using the full table of DE results to get what I call the GSEA result using clusterProfiler (Yu (n.d.)). In the following block I will use the ‘all_cprofiler’ function on the data structures named ‘p8_retina_tables’ and ‘opposite_p8_retina_tables’ in order to get these GSEA results for each contrast performed (het/wt, ko/wt, het/ko). I will follow that up with ‘all_gprofiler’ which does the same, but uses gProfiler’s enrichment analyses (it will therefore include what we just looked at).

p08_retina_all_cp <- all_cprofiler(p8_retina_sig, p8_retina_tables, orgdb = "org.Mm.eg.db")

## Error in loadNamespace(orgdb) : there is no package called 'org.Mm.eg.db'
## Error in loadNamespace(orgdb) : there is no package called 'org.Mm.eg.db'

## Error in `simple_cl[["kegg_universe"]]`:
## ! subscript out of bounds

enrichplot::dotplot(p08_retina_all_cp[["ko_vs_het_up"]][["enrich_objects"]][["MF_all"]])

## Error in `h()`:
## ! error in evaluating the argument 'object' in selecting a method for function 'dotplot': object 'p08_retina_all_cp' not found

p08_topn_gsea <- plot_topn_gsea(p08_retina_all_cp)

## Error in `h()`:
## ! error in evaluating the argument 'gse' in selecting a method for function 'plot_topn_gsea': object 'p08_retina_all_cp' not found

pp(file = "images/gsea_p08_retina_ko_vs_het_top_hit.png")
p08_topn_gsea[["GO_ko_vs_het_up"]][[1]]

## Error:
## ! object 'p08_topn_gsea' not found

plotted <- dev.off()
p08_topn_gsea[["GO_ko_vs_het_up"]][[1]]

## Error:
## ! object 'p08_topn_gsea' not found

p08_topn_gsea[["GO_ko_vs_het_up"]][[2]]

## Error:
## ! object 'p08_topn_gsea' not found

p08_topn_gsea[["GO_ko_vs_het_up"]][[3]]

## Error:
## ! object 'p08_topn_gsea' not found

p08_topn_gsea[["GO_ko_vs_het_up"]][[4]]

## Error:
## ! object 'p08_topn_gsea' not found

p08_topn_gsea[["GO_ko_vs_het_up"]][[5]]

## Error:
## ! object 'p08_topn_gsea' not found

#gsea_ko <-  gsea_result_ko[["result"]] %>%
#    select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
#    arrange(desc(recall)) %>%
#    head(n = 10)
#  gsea_plots_ko <- ggplot(gsea_ko, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
#  geom_bar(stat = "identity")+
#  scale_fill_continuous(low = "blue", high = "red") +
#  theme_bw()+
#  ylab("") +
#  xlab("GSEA Score")
gsea_het <-  gsea_result_het[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 10)

## Error in `UseMethod()`:
## ! no applicable method for 'select' applied to an object of class "NULL"

gsea_plots_het <- ggplot(gsea_het, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over Representation Score")

## Error:
## ! object 'gsea_het' not found

pp(file = "images/GSEA_p08_axontrap_retinahet_upregulated_vs_retinako.pdf")
gsea_plots_het

## Error:
## ! object 'gsea_plots_het' not found

plotted <- dev.off()

gsea_all <-  gsea_result_alldysregulated[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 10)
gsea_plots_all <- ggplot(gsea_all, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over Representation Score")

pp(file = "images/GSEA_p08_retina_axontrap_alldysregulatedgenes.pdf")
gsea_plots_all
plotted <- dev.off()

14 SCN Het vs KO

It is only now that I realized we are splitting the data by location for each set of comparisons. I think that, left to my own devices, I would prefer to keep the input data structure intact, perform the somewhat larger number of contrasts, and then split up the results. Ideally this will slightly improve the fidelity of the results returned by DESeq2 and friends. But, I will run the state of Theresa’s notebook with as few changes as possible first, then add this.

14.1 PCA

I am going to skip this PCA plot for a couple of reasons: I already did a superset of it, and the subset Theresa performed is not valid given the set of samples included in my sample sheet, and figuring out the actually corresponding subset will take me forever… In addition, I want to use my mm38_hisat_v3 for everything…

mm38_subset <- subset_se(
  mm38_hisat,
  subset = "(batch == '4' | batch == '5' | batch == '6') & time == 'p08' & location == 'scn' | sampleid == 'iprgc_03'")
mm38_norm <- normalize(mm38_subset, filter = TRUE, convert = "cpm",
                            transform = "log2", batch = "svaseq")
mm38_norm <- set_batches(mm38_norm, fact = "location")
mm38_norm <- set_conditions(mm38_norm, fact = "genotype")
pca_norm <- plot_pca(mm38_norm, max_overlaps = 70)
pca_norm$plot

Instead I will simplify the subset and see what happens…

scn_samples <- subset_se(mm38_hisat_v3,
                           subset = "location_atb == 'scn'") %>%
  set_batches(fact = "location_atb") %>%
  set_conditions(fact = "genotype_atb", colors = color_choices[["genotype"]])

## The number of samples by batch are:

## 
## scn 
##  17

## The numbers of samples by condition are:

## 
## het  ko  wt 
##   6   6   5

scn_norm <- normalize(scn_samples, filter = TRUE, convert = "cpm",
                           transform = "log2", batch = "svaseq")

## Removing 11109 low-count genes (14316 remaining).

## transform_counts: Found 919 values less than 0.

## transform_counts: Found 919 values equal to 0, adding 1 to the matrix.

scn_norm_pca <- plot_pca(scn_norm)
scn_norm_pca

## The result of performing a fast_svd dimension reduction.
## The x-axis is PC1 and the y-axis is PC2
## Colors are defined by het, ko, wt
## Shapes are defined by scn.

15 Library sizes post-deduplication

Theresa’s next operation was to perform libsize/nonzero plots. I already did the pre/post deduplication nonzero, here is the analagous libsize.

v2 is pre-deduplication and v3 is post.

plot_libsize(mm38_hisat_v2)

## $plot

## 
## $table
##            id      sum  condition  colors
##        <char>    <num>     <fctr>  <char>
##  1:  iprgc_62  5642632   het_dlgn #E7298A
##  2:  iprgc_63  3717242   het_dlgn #E7298A
##  3:  iprgc_64 12978365 het_retina #238B45
##  4:  iprgc_65 14696641 het_retina #238B45
##  5:  iprgc_66  8272382    het_scn #4292C6
##  6:  iprgc_67  7908083    het_scn #4292C6
##  7:  iprgc_68  4602849    ko_dlgn #C994C7
##  8:  iprgc_69 18774803  ko_retina #74c476
##  9:  iprgc_70  5477804     ko_scn #9BCAE1
## 10:  iprgc_71  6549329    wt_dlgn #980043
## 11:  iprgc_72  7357963    wt_dlgn #980043
## 12:  iprgc_73  7125338    wt_dlgn #980043
## 13:  iprgc_74 15075002  wt_retina #004008
## 14:  iprgc_75 14804383  wt_retina #004008
## 15:  iprgc_76 22571664  wt_retina #004008
## 16:  iprgc_77  5038166     wt_scn #08519C
## 17:  iprgc_81  8652212    wt_dlgn #980043
## 18:  iprgc_82  9606265    wt_dlgn #980043
## 19:  iprgc_83 13230777  wt_retina #004008
## 20:  iprgc_84 13312390  wt_retina #004008
## 21:  iprgc_85  6910252     ko_scn #9BCAE1
## 22:  iprgc_86 14273411  ko_retina #74c476
## 23:  iprgc_87  5647414    ko_dlgn #C994C7
## 24:  iprgc_88  7865435   het_dlgn #E7298A
## 25:  iprgc_89 11611851    het_scn #4292C6
## 26:  iprgc_90 23919491 het_retina #238B45
## 27:  iprgc_91 16016160  wt_retina #004008
## 28:  iprgc_92 10312147    wt_dlgn #980043
## 29:  iprgc_93  8088854     wt_scn #08519C
## 30:  iprgc_94  7103946   het_dlgn #E7298A
## 31:  iprgc_95  7471948    het_scn #4292C6
## 32:  iprgc_96 11278102 het_retina #238B45
## 33:  iprgc_97 12312129     wt_scn #08519C
## 34:  iprgc_98 14054244    wt_dlgn #980043
## 35:  iprgc_99 17661585  wt_retina #004008
## 36: iprgc_100 12256388    ko_dlgn #C994C7
## 37: iprgc_101 12622526  ko_retina #74c476
## 38: iprgc_102 18809254     ko_scn #9BCAE1
## 39: iprgc_104 11375862   het_dlgn #E7298A
## 40: iprgc_105 16846374   het_dlgn #E7298A
## 41: iprgc_106 17994083   het_dlgn #E7298A
## 42: iprgc_107  8231918    ko_dlgn #C994C7
## 43: iprgc_108 18552934    ko_dlgn #C994C7
## 44: iprgc_109 24270138  wt_retina #004008
## 45: iprgc_110 13509357    het_scn #4292C6
## 46: iprgc_111 19606417    het_scn #4292C6
## 47: iprgc_112 20911855    het_scn #4292C6
## 48: iprgc_113 12217700     ko_scn #9BCAE1
## 49: iprgc_114 19402528     ko_scn #9BCAE1
## 50: iprgc_115 19560965  wt_retina #004008
## 51: iprgc_116 20964695  wt_retina #004008
## 52: iprgc_117 18276778 het_retina #238B45
## 53: iprgc_118 20885024 het_retina #238B45
## 54: iprgc_119 16774668 het_retina #238B45
## 55: iprgc_120 24538069  ko_retina #74c476
## 56: iprgc_121 19040231  ko_retina #74c476
## 57: iprgc_122 21808522  ko_retina #74c476
## 58: iprgc_123  7015938    ko_dlgn #C994C7
## 59: iprgc_124  8986063     ko_scn #9BCAE1
## 60: iprgc_125 19243970    wt_dlgn #980043
## 61: iprgc_126 18681748    wt_dlgn #980043
## 62: iprgc_127 20184962    wt_dlgn #980043
## 63: iprgc_128 18172993     wt_scn #08519C
## 64: iprgc_129 17955093     wt_scn #08519C
## 65: iprgc_130 23057879     wt_scn #08519C
##            id      sum  condition  colors
##        <char>    <num>     <fctr>  <char>
## 
## $summary
##     condition      min      1st   median     mean      3rd      max
##        <fctr>    <num>    <num>    <num>    <num>    <num>    <num>
## 1:   het_dlgn  3717242  6373289  7865435 10077939 14111118 17994083
## 2: het_retina 11278102 13837503 16774668 16972724 19580901 23919491
## 3:    het_scn  7471948  8090232 11611851 12755985 16557887 20911855
## 4:    ko_dlgn  4602849  5989545  7623928  9384574 11250270 18552934
## 5:  ko_retina 12622526 15398759 18907517 18509594 21116449 24538069
## 6:     ko_scn  5477804  7429205 10601882 11967267 17161366 19402528
## 7:    wt_dlgn  6549329  7681525  9959206 12176818 17524872 20184962
## 8:  wt_retina 13230777 14872038 16838872 17746776 20613762 24270138
## 9:     wt_scn  5038166  9144673 15133611 14104186 18118518 23057879
## 
## attr(,"class")
## [1] "hpgltools::plot_libsize"

post_filter_nonzero <- plot_libsize(mm38_hisat_v3, text = FALSE)
pp(file = "images/post_all_filteres_nonzero.pdf")
post_filter_nonzero[["plot"]]
plotted <- dev.off()
post_filter_nonzero

## $plot

## 
## $table
##            id      sum  condition  colors
##        <char>    <num>     <fctr>  <char>
##  1:  iprgc_62  2470913   het_dlgn #E7298A
##  2:  iprgc_63  1606125   het_dlgn #E7298A
##  3:  iprgc_64  6404271 het_retina #238B45
##  4:  iprgc_65  7518843 het_retina #238B45
##  5:  iprgc_67  2187019    het_scn #4292C6
##  6:  iprgc_68  2068560    ko_dlgn #C994C7
##  7:  iprgc_69  8447842  ko_retina #74c476
##  8:  iprgc_70  1346388     ko_scn #9BCAE1
##  9:  iprgc_71  2164350    wt_dlgn #980043
## 10:  iprgc_72  2972198    wt_dlgn #980043
## 11:  iprgc_73  2866557    wt_dlgn #980043
## 12:  iprgc_74  6089919  wt_retina #004008
## 13:  iprgc_75  6753061  wt_retina #004008
## 14:  iprgc_76 10322488  wt_retina #004008
## 15:  iprgc_77  1723663     wt_scn #08519C
## 16:  iprgc_81  1264475    wt_dlgn #980043
## 17:  iprgc_82  2436292    wt_dlgn #980043
## 18:  iprgc_83  2834884  wt_retina #004008
## 19:  iprgc_84  6296709  wt_retina #004008
## 20:  iprgc_85  1757435     ko_scn #9BCAE1
## 21:  iprgc_86  6510628  ko_retina #74c476
## 22:  iprgc_87  1843510    ko_dlgn #C994C7
## 23:  iprgc_88  2582249   het_dlgn #E7298A
## 24:  iprgc_89  4600083    het_scn #4292C6
## 25:  iprgc_90 10355360 het_retina #238B45
## 26:  iprgc_91  7632161  wt_retina #004008
## 27:  iprgc_92  2533239    wt_dlgn #980043
## 28:  iprgc_93  1392414     wt_scn #08519C
## 29:  iprgc_94  2504277   het_dlgn #E7298A
## 30:  iprgc_95  2113705    het_scn #4292C6
## 31:  iprgc_96  5414821 het_retina #238B45
## 32:  iprgc_97  2966414     wt_scn #08519C
## 33:  iprgc_98  6228492    wt_dlgn #980043
## 34:  iprgc_99  8526541  wt_retina #004008
## 35: iprgc_100  4730617    ko_dlgn #C994C7
## 36: iprgc_101  6178771  ko_retina #74c476
## 37: iprgc_102  3553520     ko_scn #9BCAE1
## 38: iprgc_104  2057626   het_dlgn #E7298A
## 39: iprgc_105  3737220   het_dlgn #E7298A
## 40: iprgc_106  6928209   het_dlgn #E7298A
## 41: iprgc_107  2603060    ko_dlgn #C994C7
## 42: iprgc_108  7610541    ko_dlgn #C994C7
## 43: iprgc_109 10979038  wt_retina #004008
## 44: iprgc_110  2306153    het_scn #4292C6
## 45: iprgc_111  2897409    het_scn #4292C6
## 46: iprgc_112  5384209    het_scn #4292C6
## 47: iprgc_113  2655774     ko_scn #9BCAE1
## 48: iprgc_114  7458837     ko_scn #9BCAE1
## 49: iprgc_115  8514674  wt_retina #004008
## 50: iprgc_116  9144805  wt_retina #004008
## 51: iprgc_117  8451340 het_retina #238B45
## 52: iprgc_118  9421863 het_retina #238B45
## 53: iprgc_119  7738068 het_retina #238B45
## 54: iprgc_120 10171748  ko_retina #74c476
## 55: iprgc_121  8345914  ko_retina #74c476
## 56: iprgc_122  8757354  ko_retina #74c476
## 57: iprgc_123  2195429    ko_dlgn #C994C7
## 58: iprgc_124  2064033     ko_scn #9BCAE1
## 59: iprgc_125  3664824    wt_dlgn #980043
## 60: iprgc_126  1887626    wt_dlgn #980043
## 61: iprgc_127  5944235    wt_dlgn #980043
## 62: iprgc_128  2125423     wt_scn #08519C
## 63: iprgc_129  1644898     wt_scn #08519C
##            id      sum  condition  colors
##        <char>    <num>     <fctr>  <char>
## 
## $summary
##     condition     min     1st  median    mean     3rd      max
##        <fctr>   <num>   <num>   <num>   <num>   <num>    <num>
## 1:   het_dlgn 1606125 2264270 2504277 3126660 3159734  6928209
## 2: het_retina 5414821 6961557 7738068 7900652 8936602 10355360
## 3:    het_scn 2113705 2216802 2601781 3248096 4174414  5384209
## 4:    ko_dlgn 1843510 2100277 2399244 3508620 4198728  7610541
## 5:  ko_retina 6178771 6969450 8396878 8068710 8679976 10171748
## 6:     ko_scn 1346388 1834084 2359904 3139331 3329084  7458837
## 7:    wt_dlgn 1264475 2232336 2699898 3196229 3491668  6228492
## 8:  wt_retina 2834884 6410797 8073418 7709428 8990239 10979038
## 9:     wt_scn 1392414 1644898 1723663 1970562 2125423  2966414
## 
## attr(,"class")
## [1] "hpgltools::plot_libsize"

I am a bit concerned about some of these library sizes post-deduplication.

Let us look at the relationship between reads and duplication, which I assume will be relatively linear.

test <- colData(mm38_hisat_v3)[, c("hisat_genome_single_all", "umi_dedup_pct_reads")]
test_plot <- plot_linear_scatter(test, loess = TRUE)
test_plot[["scatter"]]

## `geom_smooth()` using formula = 'y ~ x'

## Warning: The following aesthetics were dropped during statistical transformation: label.
## i This can happen when ggplot fails to infer the correct grouping structure in
##   the data.
## i Did you forget to specify a `group` aesthetic or to convert a numerical
##   variable into a factor?

Theresa also produced a density/sample plot, that might prove quite useful for these due to their significantly larger variance across samples (due to deduplication).

mm38_density <- plot_density(loc_geno_nt)
mm38_density[["plot"]] +
  theme(legend.position = "none")

plot_boxplot(loc_geno_nt)

## Plot describing the gene distribution from a dataset.

There is some difference across sample densities, but it is not too crazytown.

16 Diverging a little

At this point in the document I read ahead a bit and came to the conclusion that it repeats the above logic of taking the union of wt comparisons to remove genes from the appropriate het/ko or p15/p08 or location comparisons. This seems quite reasonable to me, but I would prefer to not separate all the data, so I will attempt to duplicate and slightly streamline this logic on the full dataset. Thus I am going to skip down to the end and attempt to implement this.

16.1 DE

mm_de_normal_p8_scn <- all_pairwise(mm38_subset, model_batch = "svaseq",
                                    parallel = FALSE, do_ebseq = FALSE, do_basic = FALSE,
                                    do_dream = FALSE, do_noiseq = FALSE, do_edger = FALSE,
                                    filter = TRUE)
annots_to_merge <- mm_annot %>%
  select(ensembl_gene_id, external_gene_name) %>%
  filter(ensembl_gene_id %in% rownames(mm_de_normal_p8_scn$deseq$all_tables$ko_scn_vs_het_scn)) %>%
  distinct()
mm_de_normal_p8_scn$deseq$all_tables$ko_scn_vs_het_scn <- merge(
  mm_de_normal_p8_scn$deseq$all_tables$ko_scn_vs_het_scn,
  annots_to_merge, by.x = 0, by.y = "ensembl_gene_id", all.x = TRUE)

hetkeeper_genes <- mm_de_normal_p8_scn$deseq$all_tables$wt_scn_vs_het_scn %>%
  filter(logFC <= -0.1 & adj.P.Val <= 0.05)
kokeeper_genes <- mm_de_normal_p8_scn$deseq$all_tables$wt_scn_vs_ko_scn %>%
  filter(logFC <= -0.1 & adj.P.Val <= 0.05)
keepergenes <- unique(c(rownames(hetkeeper_genes), rownames(kokeeper_genes)))
df <- mm_de_normal_p8_scn$deseq$all_tables$koscn_vs_hetscn %>%
  dplyr::mutate(logFC = -logFC) %>%
  set_sig_limma(factors = c("Het Enriched",
                            "KO Enriched"))
df <- df %>%
  filter(Row.names %in% keepergenes)
labels_ups <- df %>%
  filter(abs(logFC) > 1) %>%
  arrange(logFC) %>%
  head(n = 1)
labels_downs <- df %>%
  filter(abs(logFC) > 1) %>%
  arrange(-logFC) %>%
  head(n = 1)
labels <- rbind(labels_ups, labels_downs)
res_tbl <- df
DEplot <- ggplot(res_tbl, aes(x = logFC, y = -log10(adj.P.Val), label = external_gene_name)) +
  geom_point(aes(colour = Significance), size = 4) +
  geom_vline(xintercept = c(-1, 1)) +
  geom_hline(yintercept = -log10(0.05)) +
  theme_classic(base_size = 20) +
  xlab("log2(FC)") +
  ylab("-log10(p-value)") +
  ## ggtitle(title, subtitle = subtitle) +
  theme(legend.position="right") +
  scale_color_manual(values=c("Het Enriched" = "#F8766D",
                              "KO Enriched" = "#00BFC4",
                              "Not\n Enriched" = "Grey")) +
  geom_label_repel(data=filter(df,
                               ## c('s5_het_dlgn', 's5_het_ret', 's5_het_scn')),
                               external_gene_name %in% labels$external_gene_name),
                   ## nudge_x = -0.5,
                   nudge_y = 3, max.overlaps = 15) +
  ggtitle("SCN Het vs KO Translatome")

pp(file = "images/p08_scn_DE_1312024.pdf")
DEplot
plotted <- dev.off()

writexl::write_xlsx(df, path = "excel/scnhet_vs_scnko_WTfiltered.xlsx")

16.2 How many ups/downs

ko_enriched <- df %>%
  filter(Significance == "KO Enriched")

het_enriched <- df %>%
  filter(Significance == "Het Enriched")

16.3 GSEA

ko_genes <- res_tbl %>%
  filter(adj.P.Val <= 0.05) %>%
  arrange(-abs(logFC)) %>%
  select(Row.names, logFC, adj.P.Val, external_gene_name, Significance) %>%
  filter(logFC <= -1)

het_genes <- res_tbl %>%
  filter(adj.P.Val <= 0.05) %>%
  arrange(-abs(logFC)) %>%
  select(Row.names, logFC, adj.P.Val, external_gene_name, Significance) %>%
  filter(logFC >= 1)

alldysregulated_genes <- res_tbl %>%
  filter(adj.P.Val <= 0.05) %>%
  arrange(logFC) %>%
  select(Row.names, logFC, adj.P.Val, external_gene_name, Significance) %>%
  filter(abs(logFC) >= 1)

gsea_result_ko <- gost(query = ko_genes$external_gene_name,
                       organism = "mmusculus",
                       evcodes = TRUE,
                       ordered_query = TRUE)

gsea_result_het <- gost(query = het_genes$external_gene_name,
                        organism = "mmusculus",
                        evcodes = TRUE,
                        ordered_query = TRUE)

gsea_result_alldysregulated <- gost(query = alldysregulated_genes$external_gene_name,
                                    organism = "mmusculus",
                                    evcodes = TRUE,
                                    ordered_query = TRUE)

gsea_ko <-  gsea_result_ko[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 10)
gsea_plots_ko <- ggplot(gsea_ko, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over enrichment Score")

gsea_het <-  gsea_result_het[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 10)
gsea_plots_het <- ggplot(gsea_het, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over enrichment Score")

gsea_all <-  gsea_result_alldysregulated[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 10)
gsea_plots_all <- ggplot(gsea_all, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over enrichment Score")

pp(file = "images/GSEA_p08_retina_axontrap_alldysregulatedgenes.pdf")
gsea_plots_all
plotted <- dev.off()

17 Het Retina vs Het SCN

mm38_subset2 <- subset_se(
  mm38_hisat,
  subset = "(batch == '4' | batch == '5' | batch == '6') & time == 'p08' & genotype != 'ko' & location != 'dlgn' | sampleid == 'iprgc_03'")

mm38_subset2 <- subset_se(mm38_subset2, subset = "sampleid != 'iprgc_89'")
mm38_subset2$design %>%
  select(genotype, location) %>%
  table()
mm38_norm2 <- normalize(mm38_subset2, filter=TRUE,
                             convert="cpm",
                             transform="log2", batch = "svaseq")

17.1 PCA

mm38_norm2 <- set_batches(mm38_norm2, fact = "location")
mm38_norm2 <- set_conditions(mm38_norm2, fact = "genotype")
pca_norm2 <- plot_pca(mm38_norm2, max_overlaps = 70)
pca_norm2$plot

17.2 DE

mm_de_subset2 <- all_pairwise(mm38_subset2,
                              model_batch="svaseq",
                              parallel=FALSE, do_ebseq=FALSE,
                              do_basic = FALSE, do_dream = FALSE,
                              do_noiseq = FALSE, do_edger = FALSE,
                              filter = TRUE)

retinakeeper_genes <- mm_de_subset2$deseq$all_tables$wt_retina_vs_het_retina %>%
  filter(logFC <= -0.1 & adj.P.Val <= 0.05)

scnkeeper_genes <- mm_de_subset2$deseq$all_tables$wt_scn_vs_het_scn %>%
  filter(logFC <= -0.1 & adj.P.Val <= 0.05)

keepergenes <- unique(c(rownames(retinakeeper_genes), rownames(scnkeeper_genes)))

annots_to_merge <- mm_annot %>%
  select(ensembl_gene_id, external_gene_name) %>%
  filter(ensembl_gene_id %in% rownames(mm_de_subset2$deseq$all_tables$het_scn_vs_het_retina)) %>%
  distinct()

mm_de_subset2$deseq$all_tables$het_scn_vs_het_retina <- merge(
  mm_de_subset2$deseq$all_tables$het_scn_vs_het_retina,
  annots_to_merge, by.x = 0,
  by.y = "ensembl_gene_id", all.x = TRUE)

df <- mm_de_subset2$deseq$all_tables$het_scn_vs_het_retina %>%
  mutate(Significance = case_when(logFC <= -1.0 ~ "Retina Enriched",
                                  logFC >= 1.0 ~ "SCN Enriched",
                                  logFC > -1.0 & logFC < 1.0 ~ "Not\n Enriched"))

df <- df %>%
  filter(Row.names %in% keepergenes)

scn_enriched <- df %>%
  filter(adj.P.Val <= 0.05 & logFC >= 1.0) %>%
  arrange(-logFC) %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val) %>%
  mutate(Significance = "SCN Enriched") %>%
  filter(Row.names %in% rownames(scnkeeper_genes))

retina_enriched <- df %>%
  filter(adj.P.Val <= 0.05 & logFC <= -1.0) %>%
  arrange(logFC)  %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val) %>%
  mutate(Significance = "Retina Enriched") %>%
  filter(Row.names %in% rownames(retinakeeper_genes))

notenriched <- df %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val, Significance) %>%
  filter(Row.names %in% c(rownames(retinakeeper_genes),
                          rownames(scnkeeper_genes))[duplicated(c(rownames(retinakeeper_genes),
                                                                  rownames(scnkeeper_genes)))]) %>%
  filter(Significance == "Not\n Enriched")

df <- rbind(scn_enriched, retina_enriched, notenriched)
df <- df %>%
  distinct()

## writexl::write_xlsx(df, path = "axonTRAP_DE_results_20240202/retinahet_vs_scn_het_WTfiltered.xlsx")

labels_ups <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1.0) %>%
  arrange(logFC) %>%
  head(n = 10)

labels_downs <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1.0) %>%
  arrange(-logFC) %>%
  head(n = 10)

labels <- rbind(labels_ups, labels_downs)

labels_requested <- c("Cdh10","Cdh12","Cdh13","Cdh18",
                      "Cdh7","Cdh8","Cdh9","Cntn3",
                      "Cntn4","Cntn5","Cntn6","Kirrel3",
                      "Nrxn1","Nrxn3","Sema3c","Sema6d",
                      "Tenm1","Tenm2","Tenm4")
res_tbl <- df
DEplot <- ggplot(res_tbl, aes(x = logFC, y = -log10(adj.P.Val), label = external_gene_name)) +
  geom_point(aes(colour = Significance), size = 4) +
  geom_vline(xintercept = c(-1, 1)) +
  geom_hline(yintercept = -log10(0.05)) +
  theme_classic(base_size = 20) +
  xlab("log2(FC)") +
  ylab("-log10(p-value)") +
  ## ggtitle(title, subtitle = subtitle) +
  theme(legend.position="right") +
  scale_color_manual(values=c("Grey", "#F8766D", "#00BFC4")) +
  geom_label_repel(data=filter(df,
                               external_gene_name %in% labels_requested),
                   ## c(labels$external_gene_name, "Opn4")), #c('s5_het_dlgn', 's5_het_ret', 's5_het_scn')),
                   ## nudge_x = -0.5,
                   nudge_y = 15, max.overlaps = 25)

#pp(file = "axonTRAP_Volcanoplots_20240202/p08_retinavsscnhet_DE_requested_genelabels_02052024.pdf")
DEplot
#dev.off()

17.3 How many ups/downs

scn_enriched <- df %>%
  filter(adj.P.Val <= 0.05 & logFC >= 1.0) %>%
  arrange(-logFC) %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val, Significance)

retina_enriched <- df %>%
  filter(adj.P.Val <= 0.05 & logFC <= -1.0) %>%
  arrange(logFC)  %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val, Significance)

scn_enriched
retina_enriched

df %>%
  filter(Significance == "Not\n Enriched")

17.4 GSEA

gsea_result_scn <- gost(query = scn_enriched$external_gene_name,
                        organism = "mmusculus", evcodes = TRUE,
                        ordered_query = TRUE, source = c("GO"))

gsea_result_ret <- gost(query = retina_enriched$external_gene_name,
                        organism = "mmusculus", evcodes = TRUE,
                        ordered_query = TRUE, source = c("GO"))

gsea_scn <-  gsea_result_scn[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 20)
gsea_plots_scn <- ggplot(gsea_scn, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over enrichment Score")

pp(file = "images/GSEA_SCNhet_vs_retina_enriched_P08.pdf")
gsea_plots_scn
plotted <- dev.off()

gsea_ret <-  gsea_result_ret[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 20)
gsea_plots_ret <- ggplot(gsea_ret, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("Over enrichment Score")

pp(file = "images/GSEA_Retinahet_vs_SCN_enriched_P08.pdf")
gsea_plots_ret
plotted <- dev.off()

18 KO Retina vs KO SCN

mm38_subset3 <- subset_se(
  mm38_hisat,
  subset = "(batch == '4' | batch == '5' | batch == '6') & time == 'p08'  & genotype != 'het' & location != 'dlgn' | sampleid == 'iprgc_03'")

mm38_subset3 <- subset_se(mm38_subset3, subset = "sampleid != 'iprgc_86'")
mm38_subset3$design %>%
  select(genotype, location) %>%
  table()
mm38_norm3 <- normalize(mm38_subset3, filter=TRUE,
                             convert="cpm", transform="log2", batch = "svaseq")

18.1 PCA

mm38_norm3 <- set_batches(mm38_norm3, fact = "location")
mm38_norm3 <- set_conditions(mm38_norm3, fact = "genotype")
pca_norm3 <- plot_pca(mm38_norm3, max_overlaps = 70)
pca_norm3$plot

18.2 DE

mm_de_subset3 <- all_pairwise(mm38_subset3,
                              model_batch="svaseq",
                              parallel=FALSE, do_ebseq=FALSE,
                              do_basic = FALSE, do_dream = FALSE,
                              do_noiseq = FALSE, do_edger = FALSE,
                              filter = TRUE)

retinakeeper_genes <- mm_de_subset3$deseq$all_tables$wtretina_vs_koretina %>%
  filter(logFC <= -1.0 & adj.P.Val <= 0.05)

scnkeeper_genes <- mm_de_subset3$deseq$all_tables$wtscn_vs_koscn %>%
  filter(logFC <= -1.0 & adj.P.Val <= 0.05)

keepergenes <- unique(c(rownames(retinakeeper_genes), rownames(scnkeeper_genes)))

annots_to_merge <- mm_annot %>%
  select(ensembl_gene_id, external_gene_name) %>%
  filter(ensembl_gene_id %in% rownames(mm_de_subset3$deseq$all_tables$ko_scn_vs_ko_retina)) %>%
  distinct()

mm_de_subset3$deseq$all_tables$ko_scn_vs_ko_retina <- merge(
  mm_de_subset3$deseq$all_tables$ko_scn_vs_ko_retina,
  annots_to_merge, by.x = 0,
  by.y = "ensembl_gene_id", all.x = TRUE)

df <- mm_de_subset3$deseq$all_tables$ko_scn_vs_ko_retina %>%
  mutate(Significance = case_when(logFC <= -1 ~ "Retina Enriched",
                                  logFC >= 1 ~ "SCN Enriched",
                                  logFC > -1 & logFC < 1 ~ "Not\n Enriched"))

df <- df %>%
  filter(Row.names %in% keepergenes)

scn_enriched <- df %>%
  filter(adj.P.Val <= 0.05 & logFC >= 1) %>%
  arrange(-logFC) %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val) %>%
  mutate(Significance = "SCN Enriched") %>%
  filter(Row.names %in% rownames(scnkeeper_genes))

df %>%
  filter(adj.P.Val <= 0.05 & logFC <= -1) %>%
  arrange(logFC)  %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val) %>%
  mutate(Significance = "Retina Enriched") %>%
  filter(Row.names %in% rownames(retinakeeper_genes)) -> retina_enriched

notenriched <- df %>%
  select(Row.names, external_gene_name, logFC, adj.P.Val, Significance) %>%
  filter(Row.names %in% c(rownames(retinakeeper_genes),
                          rownames(scnkeeper_genes))[duplicated(c(rownames(retinakeeper_genes),
                                                                  rownames(scnkeeper_genes)))])

df <- rbind(scn_enriched, retina_enriched, notenriched)

labels_ups <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1) %>%
  arrange(logFC) %>%
  head(n = 10)

labels_downs <- df %>%
  filter(adj.P.Val <= 0.05 & abs(logFC) > 1) %>%
  arrange(-logFC) %>%
  head(n = 10)

labels <- rbind(labels_ups, labels_downs)
## wanted_column <- "Significance"

res_tbl <- df
DEplot <- ggplot(res_tbl, aes(x = logFC, y = -log10(adj.P.Val), label = external_gene_name)) +
  geom_point(aes(colour = Significance), size = 4) +
  ## geom_point(aes(colour = !!sym(wanted_column)), size = 4) +
  geom_vline(xintercept = c(-1, 1)) +
  geom_hline(yintercept = -log10(0.05)) +
  theme_classic(base_size = 20) +
  xlab("log2(FC)") +
  ylab("-log10(p-value)") +
  ## ggtitle(title, subtitle = subtitle) +
  theme(legend.position = "right") +
  scale_color_manual(values = c("Grey", "#F8766D", "#00BFC4")) +
  geom_label_repel(data = filter(
    df, external_gene_name %in% c(labels$external_gene_name, "Opn4")),
    ## c('s5_het_dlgn', 's5_het_ret', 's5_het_scn')),
    ## nudge_x = -0.5,
    nudge_y = 10, max.overlaps = 25)

pp(file = "images/p08_retinavsscnko_DE_1312024.pdf")
DEplot
plotted <- dev.off()

18.3 How many ups/downs

scn_enriched
retina_enriched
notenriched %>%
  filter(Significance == "Not\n Enriched")

18.4 GSEA

gsea_result_scn <- gost(query = scn_enriched$external_gene_name,
                        organism = "mmusculus",
                        evcodes = TRUE,
                        ordered_query = TRUE,
                        source = c("GO"))

gsea_result_ret <- gost(query = retina_enriched$external_gene_name,
                        organism = "mmusculus",
                        evcodes = TRUE,
                        ordered_query = TRUE,
                        source = c("GO"))

gsea_scn <-  gsea_result_scn[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 20)
gsea_plots_scn <- ggplot(gsea_scn, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("GSEA Score")

pp(file = "images/GSEA_SCNko_enriched_vs_retina_P08.pdf")
gsea_plots_scn
plotted <- dev.off()

gsea_ret <-  gsea_result_ret[["result"]] %>%
  select(term_name, p_value, term_size, intersection_size, recall, source, intersection) %>%
  arrange(desc(recall)) %>%
  head(n = 20)
gsea_plots_ret <- ggplot(gsea_ret, aes(x = recall, y = reorder(term_name, recall), fill = p_value)) +
  geom_bar(stat = "identity") +
  scale_fill_continuous(low = "blue", high = "red") +
  theme_bw() +
  ylab("") +
  xlab("GSEA Score")

pp(file = "images/GSEA_Retinako_enriched_vs_SCN_P08.pdf")
gsea_plots_ret
plotted <- dev.off()

19 My version of the global analysis

I want to have an invocation of all_pairwise() which uses all samples, in the following block I will set that up using a set of ‘keepers’ which will be named by time, location, then 2 letters for the numerator/denominator: w for WT, h for het, d for delta; thus “p08_retina_hw” is comparing the het/wt for the p08 retina samples.

If they are of interest, I will have a separate set which follows the same convention with names like “p08_ko_sr” to compare p08 deltas with SCN as the numerator and retina as the denominator.

19.1 Set up the exclusion dataset

The most peculiar aspect of this analysis resides in the choices around choosing which genes to consider when comparing the genotypes/locations/times. The general idea is pretty clear: find the genes which are non-specifically being pulled down in the WT samples and either exclude or discount them. The various potential methods for performing this are confusing:

1. Which set of comparisons of wt/ko wt/het do we use to exclude/discount genes?
   1. Should it be a combination of all samples wt vs. x?
   2. Should it be only the ‘relevant’ comparison, e.g. if we are comparing p08_dlgn_het vs. p08_scn_het; do we remove genes observed in (p08_dlgn_het/wt && p08_scn_het/wt)
2. Do we instead attempt to use this x/wt information to normalize the expression values in the other conditions and keep those genes?

Theresa’s current worksheet implements a version of 1b in which she separated the various input gene sets to define the exclusion genes. I am going to repeat this, but leave the starting data structure intact.

In this first iteration, I will do that by creating a simplified model of the data which combines the time/genotype/location and using sva. In my next iteration I will use a full statistical model containing each of those factors (and probably also using sva).

Note: my color choices are kind of garbage.

In addition, the exclusion dataset is the same as the analysis dataset, it is really only the contrasts which will be different.

v3_pairwise_input <- set_conditions(mm38_hisat_v3, fact = "time_geno_loc",
                                    colors = color_choices[["all"]])

## The numbers of samples by condition are:

## 
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

## Warning in set_se_colors(new_se, colors = colors): Colors for the following
## categories are not being used: p60_wt_dlgn, p60_wt_retina, p60_wt_scn.

19.2 Set up the contrasts

In the following few blocks I will set up the various comparisons of interest. Starting with the set of genes to exclude because they were observed to bind non-specifically in the wt samples.

19.2.1 Inclusion contrasts

In each exclusion I will have the contrast first followed by the pair of contrasts which will be used to define the gene set to exclude.

• p15_het_dlgn/p08_het_dlgn: p15_wt_dlgn/p15_het_dlgn, p08_wt_dlgn/p08_het_dlgn; remove the genes increased in wt.
• p15_ko_scn/p08_ko_scn: p15_wt_scn/p15_ko_scn, p08_wt_scn/p15_ko_scn
• p15_het_retina/p08_het_retina: I think you get it, wt/het for both p15 retinas and p08 retinas…

Put slightly differently, for every term of interest I will create a contrast with the wt as numerator and the desired term as denominator, then pull out the genes increased in wt.

inclusions <- list(
  ## I like alphabetizing things, start with dlgn
  "p15_het_dlgn" = c("p15_het_dlgn", "p15_wt_dlgn"),
  "p08_het_dlgn" = c("p08_het_dlgn", "p08_wt_dlgn"),
  "p15_ko_dlgn" = c("p15_ko_dlgn", "p15_wt_dlgn"),
  "p08_ko_dlgn" = c("p08_ko_dlgn", "p08_wt_dlgn"),
  ## Then retinas
  "p15_het_retina" = c("p15_het_retina", "p15_wt_retina"),
  "p08_het_retina" = c("p08_het_retina", "p08_wt_retina"),
  "p15_ko_retina" = c("p15_ko_retina", "p15_wt_retina"),
  "p08_ko_retina" = c("p08_ko_retina", "p08_wt_retina"),
  ## Then scn
  "p15_het_scn" = c("p15_het_scn", "p15_wt_scn"),
  "p08_het_scn" = c("p08_het_scn", "p08_wt_scn"),
  "p15_ko_scn" = c("p15_ko_scn", "p15_wt_scn"),
  "p08_ko_scn" = c("p08_ko_scn", "p08_wt_scn"))

19.2.2 Time contrasts

For each location/genotype of interest, let us compare p15/p08

time_keepers <- list(
  ## DLGN
  "t_het_dlgn" = c("p15_het_dlgn", "p08_het_dlgn"),
  "t_ko_dlgn" = c("p15_ko_dlgn", "p08_ko_dlgn"),
  ## Retina
  "t_het_retina" = c("p15_het_retina", "p08_het_retina"),
  "t_ko_retina" = c("p15_ko_retina", "p08_ko_retina"),
  ## SCN
  "t_het_scn" = c("p15_het_scn", "p08_het_scn"),
  "t_ko_scn" = c("p15_ko_scn", "p08_ko_scn"))

19.2.3 Location contrasts

Compare locations and keep time/genotype consistent. I will use the location initials to define numerator/denominator.

location_keepers <- list(
  ## dlgn/retina
  "dr_p08_het" = c("p08_het_dlgn", "p08_het_retina"),
  "dr_p15_het" = c("p15_het_dlgn", "p15_het_retina"),
  "dr_p08_ko" = c("p08_ko_dlgn", "p08_ko_retina"),
  "dr_p15_ko" = c("p15_ko_dlgn", "p15_ko_retina"),
  ## scn/retina
  "sr_p08_het" = c("p08_het_scn", "p08_het_retina"),
  "sr_p15_het" = c("p15_het_scn", "p15_het_retina"),
  "sr_p08_ko" = c("p08_ko_scn", "p08_ko_retina"),
  "sr_p15_ko" = c("p15_ko_scn", "p15_ko_retina"),
  ## dlgn/scn
  "ds_p08_het" = c("p08_het_dlgn", "p08_het_scn"),
  "ds_p15_het" = c("p15_het_dlgn", "p15_het_scn"),
  "ds_p08_ko" = c("p08_ko_dlgn", "p08_ko_scn"),
  "ds_p15_ko" = c("p15_ko_dlgn", "p15_ko_scn"))

19.2.4 Genotype contrasts

Compare ko/het while keeping time/location constant. Similarly, use the initials to denote numerator/denominator, which will always be kh.

genotype_keepers <- list(
  ## DLGN
  "kh_p08_dlgn" = c("p08_ko_dlgn", "p08_het_dlgn"),
  "kh_p15_dlgn" = c("p15_ko_dlgn", "p15_het_dlgn"),
  ## Retina
  "kh_p08_retina" = c("p08_ko_retina", "p08_het_retina"),
  "kh_p15_retina" = c("p15_ko_retina", "p15_het_retina"),
  ## SCN
  "kh_p08_scn" = c("p08_ko_scn", "p08_het_scn"),
  "kh_p15_scn" = c("p15_ko_scn", "p15_het_scn"))

19.3 Perform the exclusion comparison

My all_pairwise() function now has a parameter which allows me to choose which contrasts to perform instead of literally doing every possible comparison. That is well suited for these operations:

In a container, the following appears to fail with:

“error code 1 from Lapack routine ‘dgesdd’”

Running it manually outside the container results in it working without error. I assume therefore that the problem lies in the compilation flags of LAPACK in the container.

lfc_cutoff <- 0.1
adjp_cutoff <- 0.1
inclusion_de <- all_pairwise(v3_pairwise_input, filter = "simple",
                             keepers = inclusions, model_svs = "svaseq",
                             model_fstring = "~ 0 + condition")

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

## Removing 5517 low-count genes (19908 remaining).

## Basic step 0/3: Normalizing data.

## Basic step 0/3: Converting data.

## I think this is failing? SummarizedExperiment

## Basic step 0/3: Transforming data.

## Setting 394442 entries to zero.

## This received a matrix of SVs.

## converting counts to integer mode

## gene-wise dispersion estimates

## mean-dispersion relationship

## final dispersion estimates

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

inclusion_de

## A pairwise differential expression with results from: basic, deseq, ebseq, edger, limma, noiseq.

## This used a surrogate/batch estimate from: svaseq.

## The primary analysis performed 12 comparisons.

inclusion_tables <- combine_de_tables(
  inclusion_de, keepers = inclusions, label_column = label_column,
  excel = glue("wt_comparisons/inclusion_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

inclusion_tables

## A set of combined differential expression results.

##                              table deseq_sigup deseq_sigdown edger_sigup
## 1      p15_het_dlgn_vs_p15_wt_dlgn         536           971         668
## 2      p08_het_dlgn_vs_p08_wt_dlgn          15            79          57
## 3       p15_ko_dlgn_vs_p15_wt_dlgn         903          1333        1029
## 4       p08_ko_dlgn_vs_p08_wt_dlgn         117           282         133
## 5  p15_het_retina_vs_p15_wt_retina         151            58         193
## 6  p08_het_retina_vs_p08_wt_retina         432           110         464
## 7   p15_ko_retina_vs_p15_wt_retina         107            34         178
## 8   p08_ko_retina_vs_p08_wt_retina         533           155         551
## 9        p15_het_scn_vs_p15_wt_scn          11             8          39
## 10       p08_het_scn_vs_p08_wt_scn          48             8          57
## 11        p15_ko_scn_vs_p15_wt_scn           7             5          28
## 12        p08_ko_scn_vs_p08_wt_scn          31            26          83
##    edger_sigdown limma_sigup limma_sigdown
## 1           1013         582           833
## 2            107          40            46
## 3           1346         837          1045
## 4            297         186           356
## 5            114         119            57
## 6            162         315            88
## 7             62          39            25
## 8            224         404           144
## 9             28          19            34
## 10            45          42            20
## 11            30          39            39
## 12            54          65            55

## Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
## i Please use tidy evaluation idioms with `aes()`.
## i See also `vignette("ggplot2-in-packages")` for more information.
## i The deprecated feature was likely used in the UpSetR package.
##   Please report the issue to the authors.
## This warning is displayed once per session.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

## Warning: The `size` argument of `element_line()` is deprecated as of ggplot2 3.4.0.
## i Please use the `linewidth` argument instead.
## i The deprecated feature was likely used in the UpSetR package.
##   Please report the issue to the authors.
## This warning is displayed once per session.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

## Plot describing unique/shared genes in a differential expression table.

inclusion_sig <- extract_significant_genes(
  inclusion_tables, lfc = lfc_cutoff, p = adjp_cutoff,
  according_to = "deseq",
  excel = glue("wt_comparisons/inclusion_sig-v{ver}.xlsx"))
inclusion_sig

## A set of genes deemed significant according to deseq.

## The parameters defining significant were:

## LFC cutoff: 0.1 adj P cutoff: 0.1

##                deseq_up deseq_down
## p15_het_dlgn       2067       2381
## p08_het_dlgn        180        349
## p15_ko_dlgn        2544       2633
## p08_ko_dlgn         397        572
## p15_het_retina      402        223
## p08_het_retina      976        380
## p15_ko_retina       347         93
## p08_ko_retina      1551        859
## p15_het_scn          15          9
## p08_het_scn         225         55
## p15_ko_scn            8          6
## p08_ko_scn          112        116

See the shared/unique genes in these sets.

inclusion_upsets <- upsetr_sig(inclusion_sig)
inclusion_intersects <- write_upset_groups(inclusion_upsets, excel = "excel/inclusion_gene_groups.xlsx")

19.3.1 Check vs Theresa’s filter

Up above Theresa performed a 0.25 log2FC and 0.05 adjp filter which provided a set of 2,640 genes observed higher in the p08 het retinas vs. wt retinas. I should see that in this inclusion_sig data structure.

There is an important caveat though: in Theresa’s filter above, she did a DE of only the retina samples but I did all samples. I expected that this would result in basically the same result (I actually assumed I would get a few more genes), but instead it appears to have retrieved a significantly smaller number of genes (about 1/2, happily they pretty much all appear in the previous filter). As a result, I am going to try relaxing my constraints slightly to see if I can recapitulate her filter (which would match Theresa’s later filter, though I guess that in turn will lead to a smaller set of genes compared to her later, relaxed 0.1 filter).

comparison <- inclusion_sig[["deseq"]][["ups"]][["p08_het_retina"]]
comp <- list(
  "taa" = taa_keepers,
  "new" = rownames(comparison))
test_comparison <- Vennerable::Venn(comp)
Vennerable::plot(test_comparison)

I want to have a little function which, given a contrast of interest, will extract the gene sets which should be included/excluded given the above.

write_all_cp <- function(all_cp) {
  all_written <- list()
  for (g in seq_len(length(all_cp))) {
    name <- names(all_cp)[g]
    datum <- all_cp[[name]]
    filename <- glue("cprofiler/{ver}/{name}_cprofiler-v{ver}.xlsx")
    written <- sm(write_cp_data(datum, excel = filename))
    all_written[[g]] <- written
  }
  return(all_written)
}
write_all_gp <- function(all_gp) {
  all_written <- list()
  for (g in seq_len(length(all_gp))) {
    name <- names(all_gp)[g]
    datum <- all_gp[[name]]
    filename <- glue("gprofiler/{ver}/{name}_gprofiler-v{ver}.xlsx")
    written <- sm(write_gprofiler_data(datum, excel = filename))
    all_written[[g]] <- written
  }
  return(all_written)
}
extract_inclusions <- function(inclusion_sig, inclusion_tables, inclusions, keepers, all_genes,
                               according_to = "deseq", which = "ups") {
  retlist <- list()
  table_names <- names(inclusion_sig[[according_to]][[which]])
  for (c_num in seq_along(keepers)) {
    contrast <- names(keepers)[c_num]
    numerator_name <- keepers[[c_num]][1]
    denominator_name <- keepers[[c_num]][2]
    ## In my new branch I cleaned up the sanitizer function for contrasts so this is not needed.
    ## The following two lines are no longer needed because of the cleanups I performed.
    ##numerator_name <- gsub(x = numerator_name, pattern = "(het|ko|wt)", replacement = "_\\1_")
    ##denominator_name <- gsub(x = denominator_name, pattern = "(het|ko|wt)", replacement = "_\\1_")
    numerator_table <- inclusion_sig[[according_to]][[which]][[numerator_name]]
    numerator_genes <- rownames(numerator_table)
    denominator_table <- inclusion_sig[[according_to]][[which]][[denominator_name]]
    denominator_genes <- rownames(denominator_table)
    df_columns <- paste0("deseq_", c("logfc", "adjp", "den"))
    included_num <- inclusion_tables[["data"]][[numerator_name]][, df_columns]
    colnames(included_num) <- c("numerator_vs_wt_logfc", "numerator_vs_wt_adjp", "num_wt_mean_exprs")
    included_den <- inclusion_tables[["data"]][[denominator_name]][, df_columns]
    colnames(included_den) <- c("denominator_vs_wt_logfc", "denominator_vs_wt_adjp", "den_wt_mean_exprs")
    included_df <- merge(included_num, included_den, by = "row.names")
    rownames(included_df) <- included_df[["Row.names"]]
    included_df[["Row.names"]] <- NULL
    include_genes <- unique(c(numerator_genes, denominator_genes))
    message("The set of unique genes higher in ", numerator_name,
            " vs. wt is ", length(numerator_genes), ".")
    message("The set of unique genes higher in ", denominator_name,
            " vs. wt is ", length(denominator_genes), ".")
    message("The unique union of them is ", length(include_genes), " genes.")
    include_name <- paste0("inc_", contrast)
    include_idx <- all_genes %in% include_genes
    include_genes <- all_genes[include_idx]
    df_name <- paste0("df_", contrast)
    retlist[[df_name]] <- included_df
    written_inclusion <- write_xlsx(data = included_df,
                                    excel = glue("included_genes/{include_name}-v{ver}.xlsx"))
    retlist[[include_name]] <- include_genes
    retlist[[contrast]] <- include_genes
  }
  return(retlist)
}

19.3.2 Extract genes included for each set of contrasts

Now, using that function, pull out the gene IDs of genes we do not trust because they were too high in wt for every contrast we are likely to perform.

all_genes <- rownames(exprs(v3_pairwise_input))
time_inclusions <- extract_inclusions(inclusion_sig, inclusion_tables, inclusions,
                                      time_keepers, all_genes)

## The set of unique genes higher in p15_het_dlgn vs. wt is 2067.

## The set of unique genes higher in p08_het_dlgn vs. wt is 180.

## The unique union of them is 2113 genes.

## The set of unique genes higher in p15_ko_dlgn vs. wt is 2544.

## The set of unique genes higher in p08_ko_dlgn vs. wt is 397.

## The unique union of them is 2716 genes.

## The set of unique genes higher in p15_het_retina vs. wt is 402.

## The set of unique genes higher in p08_het_retina vs. wt is 976.

## The unique union of them is 1086 genes.

## The set of unique genes higher in p15_ko_retina vs. wt is 347.

## The set of unique genes higher in p08_ko_retina vs. wt is 1551.

## The unique union of them is 1664 genes.

## The set of unique genes higher in p15_het_scn vs. wt is 15.

## The set of unique genes higher in p08_het_scn vs. wt is 225.

## The unique union of them is 238 genes.

## The set of unique genes higher in p15_ko_scn vs. wt is 8.

## The set of unique genes higher in p08_ko_scn vs. wt is 112.

## The unique union of them is 120 genes.

location_inclusions <- extract_inclusions(inclusion_sig, inclusion_tables, inclusions,
                                          location_keepers, all_genes)

## The set of unique genes higher in p08_het_dlgn vs. wt is 180.

## The set of unique genes higher in p08_het_retina vs. wt is 976.

## The unique union of them is 1134 genes.

## The set of unique genes higher in p15_het_dlgn vs. wt is 2067.

## The set of unique genes higher in p15_het_retina vs. wt is 402.

## The unique union of them is 2361 genes.

## The set of unique genes higher in p08_ko_dlgn vs. wt is 397.

## The set of unique genes higher in p08_ko_retina vs. wt is 1551.

## The unique union of them is 1883 genes.

## The set of unique genes higher in p15_ko_dlgn vs. wt is 2544.

## The set of unique genes higher in p15_ko_retina vs. wt is 347.

## The unique union of them is 2843 genes.

## The set of unique genes higher in p08_het_scn vs. wt is 225.

## The set of unique genes higher in p08_het_retina vs. wt is 976.

## The unique union of them is 1188 genes.

## The set of unique genes higher in p15_het_scn vs. wt is 15.

## The set of unique genes higher in p15_het_retina vs. wt is 402.

## The unique union of them is 417 genes.

## The set of unique genes higher in p08_ko_scn vs. wt is 112.

## The set of unique genes higher in p08_ko_retina vs. wt is 1551.

## The unique union of them is 1624 genes.

## The set of unique genes higher in p15_ko_scn vs. wt is 8.

## The set of unique genes higher in p15_ko_retina vs. wt is 347.

## The unique union of them is 355 genes.

## The set of unique genes higher in p08_het_dlgn vs. wt is 180.

## The set of unique genes higher in p08_het_scn vs. wt is 225.

## The unique union of them is 402 genes.

## The set of unique genes higher in p15_het_dlgn vs. wt is 2067.

## The set of unique genes higher in p15_het_scn vs. wt is 15.

## The unique union of them is 2080 genes.

## The set of unique genes higher in p08_ko_dlgn vs. wt is 397.

## The set of unique genes higher in p08_ko_scn vs. wt is 112.

## The unique union of them is 493 genes.

## The set of unique genes higher in p15_ko_dlgn vs. wt is 2544.

## The set of unique genes higher in p15_ko_scn vs. wt is 8.

## The unique union of them is 2550 genes.

genotype_inclusions <- extract_inclusions(inclusion_sig, inclusion_tables, inclusions,
                                          genotype_keepers, all_genes)

## The set of unique genes higher in p08_ko_dlgn vs. wt is 397.

## The set of unique genes higher in p08_het_dlgn vs. wt is 180.

## The unique union of them is 501 genes.

## The set of unique genes higher in p15_ko_dlgn vs. wt is 2544.

## The set of unique genes higher in p15_het_dlgn vs. wt is 2067.

## The unique union of them is 2773 genes.

## The set of unique genes higher in p08_ko_retina vs. wt is 1551.

## The set of unique genes higher in p08_het_retina vs. wt is 976.

## The unique union of them is 1760 genes.

## The set of unique genes higher in p15_ko_retina vs. wt is 347.

## The set of unique genes higher in p15_het_retina vs. wt is 402.

## The unique union of them is 571 genes.

## The set of unique genes higher in p08_ko_scn vs. wt is 112.

## The set of unique genes higher in p08_het_scn vs. wt is 225.

## The unique union of them is 312 genes.

## The set of unique genes higher in p15_ko_scn vs. wt is 8.

## The set of unique genes higher in p15_het_scn vs. wt is 15.

## The unique union of them is 18 genes.

19.4 Perform the DE analyses and exclude the target genes

genotype_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                            keepers = genotype_keepers, model_batch = "svaseq")

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## Basic step 0/3: Normalizing data.

## Basic step 0/3: Converting data.

## I think this is failing? SummarizedExperiment

## Basic step 0/3: Transforming data.

## Setting 109425 entries to zero.

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in variancePartition::dream(exprObj = voom_result, formula = model_fstring,  : 
##   Design matrix is singular, covariates are very correlated
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().

## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

genotype_de

## A pairwise differential expression with results from: basic, deseq, ebseq, edger, limma, noiseq.

## This used a surrogate/batch estimate from: Existing surrogate matrix.

## The primary analysis performed 6 comparisons.

location_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                            keepers = location_keepers, model_batch = "svaseq")

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## Basic step 0/3: Normalizing data.

## Basic step 0/3: Converting data.

## I think this is failing? SummarizedExperiment

## Basic step 0/3: Transforming data.

## Setting 109425 entries to zero.

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in variancePartition::dream(exprObj = voom_result, formula = model_fstring,  : 
##   Design matrix is singular, covariates are very correlated
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().
## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

location_de

## A pairwise differential expression with results from: basic, deseq, ebseq, edger, limma, noiseq.

## This used a surrogate/batch estimate from: Existing surrogate matrix.

## The primary analysis performed 12 comparisons.

time_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                        keepers = time_keepers, model_batch = "svaseq")

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## Basic step 0/3: Normalizing data.

## Basic step 0/3: Converting data.

## I think this is failing? SummarizedExperiment

## Basic step 0/3: Transforming data.

## Setting 109425 entries to zero.

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in variancePartition::dream(exprObj = voom_result, formula = model_fstring,  : 
##   Design matrix is singular, covariates are very correlated
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().
## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2

time_de

## A pairwise differential expression with results from: basic, deseq, ebseq, edger, limma, noiseq.

## This used a surrogate/batch estimate from: Existing surrogate matrix.

## The primary analysis performed 6 comparisons.

19.5 Extract the relevant tables and include genes lower in wt

19.5.1 Genotype contrasts

I will start with the tables and no inclusions so I can check my work.

In this first block I will explain a little more thoroughly what is going on:

1. Dump the full table of the contrasts I defined above comparing the 3 genotypes across time/location.
2. Iterate over each of those contrasts and do the following:
   1. Extract the name of the contrast, ‘kh_p08_dlgn’ for example
   2. Yank out that specific entry from the keeper list and its name
   3. Yank out the corresponding set of genes to include from the inclusions data structure.
   4. Create a filename given the name in (a) above and the logFC cutoff chosen for the inclusions (I am assuming we may change this)
   5. Given (b), (c), and (d), extract the corresponding table from the differential expression analysis and include the appropriate genes.

genotype_tables_full <- combine_de_tables(
  genotype_de, keepers = genotype_keepers, label_column = label_column,
  fancy = TRUE,
  excel = glue("full_contrasts/genotype_full_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

genotype_tables_full

## A set of combined differential expression results.

##                             table deseq_sigup deseq_sigdown edger_sigup
## 1     p08_ko_dlgn_vs_p08_het_dlgn           0             0           0
## 2     p15_ko_dlgn_vs_p15_het_dlgn           0             0           0
## 3 p08_ko_retina_vs_p08_het_retina           0             0           0
## 4 p15_ko_retina_vs_p15_het_retina           0             0           0
## 5       p08_ko_scn_vs_p08_het_scn           0             0           0
## 6       p15_ko_scn_vs_p15_het_scn           0             0           0
##   edger_sigdown limma_sigup limma_sigdown
## 1             0          54             1
## 2             0           0             0
## 3             0           3             1
## 4             0           0             1
## 5             0          31            10
## 6             0           0             1

## Only  has information, cannot create an UpSet.

## Plot describing unique/shared genes in a differential expression table.

## NULL

genotype_sig_full <- extract_significant_genes(
  genotype_tables_full, according_to = "deseq",
  excel = glue("full_contrasts/genotype_full_sig-v{ver}.xlsx"))

## Error in `extract_significant_genes()`:
## ! object 'do_excel' not found

genotype_sig_full

## Error:
## ! object 'genotype_sig_full' not found

genotype_full_gp <- all_gprofiler(genotype_sig_full, species = "mmusculus",
                                  excel = "excel/all_gprofiler_genotype_full.xlsx")

## Error:
## ! object 'genotype_sig_full' not found

genotype_full_cp <- all_cprofiler(genotype_sig_full, genotype_tables_full,
                                  orgdb = "org.Mm.eg.db",
                                  excel = "excel/all_cprofiler_genotype_full.xlsx")

## Error:
## ! object 'genotype_sig_full' not found

genotype_full_upset <- upsetr_sig(genotype_sig_full)

## Error:
## ! object 'genotype_sig_full' not found

genotype_full_intersects <- write_upset_groups(genotype_full_upset,
                                               excel = "excel/genotype_full_gene_groups.xlsx")

## Error:
## ! object 'genotype_full_upset' not found

genotype_tables <- list()
genotype_sig <- list()
genotype_gp <- list()
genotype_cp <- list()
for (k in seq_along(genotype_keepers)) {
  name <- names(genotype_keepers)[k]
  message("Examining ", name)
  keeper <- genotype_keepers[name]
  include_name <- paste0("inc_", name)
  include_df_name <- paste0("df_", name)
  include_df <- genotype_inclusions[[include_df_name]]
  includes <- genotype_inclusions[[include_name]]
  summary(rownames(genotype_sig_full[["deseq"]][["ups"]][[name]]) %in% includes)
  include_filename <- glue("genotype_contrasts/genotype_{name}_including_wt_{lfc_cutoff}_decreased_table-v{ver}.xlsx")
  include_sig_filename <- glue("genotype_contrasts/genotype_{name}_including_wt_{lfc_cutoff}_decreased_sig-v{ver}.xlsx")
  genotype_tables[[name]] <- combine_de_tables(
    genotype_de, extra_annot = include_df,
    keepers = keeper, label_column = label_column,
    excel = include_filename, wanted_genes = includes)
  print(genotype_tables[[name]])
  genotype_sig[[name]] <- extract_significant_genes(
    genotype_tables[[name]], according_to = "deseq",
    excel = include_sig_filename)
  print(genotype_sig[[name]])
  num_rows <- nrow(genotype_sig[[name]][["deseq"]][["ups"]][[name]]) +
    nrow(genotype_sig[[name]][["deseq"]][["downs"]][[name]])
  message("There are ", num_rows, " significant up and down genes.")
  if (num_rows >= 10) {
    message("Performing gprofiler/clusterProfiler.")
    genotype_gp[[name]] <- all_gprofiler(genotype_sig[[name]], species = "mmusculus")
    gp_written <- write_all_gp(genotype_gp[[name]])
    genotype_cp[[name]] <- all_cprofiler(genotype_sig[[name]], genotype_tables[[name]],
                                         orgdb = "org.Mm.eg.db")
    cp_written <- write_all_cp(genotype_cp[[name]])
  } else {
    warning("There are less than 10 genes up and down in the ", name, " comparison.")
    message("There are less than 10 genes up and down in the ", name, " comparison.")
  }
}

## Examining kh_p08_dlgn

## Error:
## ! object 'genotype_sig_full' not found

A few specific plots of interest: Colenso asked to label a few genes for the knockout/het p08_retinas, p08_scn, and p08_dlgn: either the top-15 or all significant. I am pretty sure if I tell it 15 and there are not that many, it will just do the significant? Let us find out!

19.5.1.1 ko/het for p08 retinas

For some crazy reason, this plot is double-labelling!

table_name <- "kh_p08_retina"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting <- c("Opn4", "Gm9008", "Lrr1", "Cnbd1")
kh_p08_retina_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp", fill = "black",
  color_low = colors[["ko_retina"]], color_high = colors[["het_retina"]],
  label_column = "mgi_symbol", label = interesting, alpha = 1.0,
  size = 4)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p08_retina_volcano.pdf", width = 9, height = 9)
kh_p08_retina_volcano[["plot"]]

## Error:
## ! object 'kh_p08_retina_volcano' not found

plotted <- dev.off()
kh_p08_retina_volcano[["plot"]]

## Error:
## ! object 'kh_p08_retina_volcano' not found

## why in the crap is it double-labelling!?

## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p08_retina_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_retina"]], color_high = colors[["het_retina"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p08_retina_ma.pdf", width = 9, height = 9)
kh_p08_retina_ma[["plot"]]

## Error:
## ! object 'kh_p08_retina_ma' not found

plotted <- dev.off()
kh_p08_retina_ma[["plot"]]

## Error:
## ! object 'kh_p08_retina_ma' not found

19.5.1.2 ko/het p08 SCN

Holy crappers, this plot did not double label; oooh I have a check in my plotter to see if there are too few/too many labels and I foolishly allowed it to concatenate the labels! What in the crap was I thinking?

I am going to make an executive decision for this plot, 15 is too many and makes it crazy cluttered.

19.5.2 Repeat this with two sets of genes

table_name <- "kh_p08_scn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting_genes <- c("Fign", "Nrn1", "Dpysl2", "Actb", "Fgf9", "Otx2", "Sec23",
                       "Ncam1", "Map4", "Sec22b", "Nlgn3", "Marcks", "Cd47",
                       "Dpysl3", "Lin7c", "Cadm1", "Snx12", "Rhoa", "Inpp5f",
                       "Atg12", "Set", "Gsk3b", "Pdcd4", "Gabra2", "Tmco1", "Anapc16")
kh_p08_scn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  label_column = "mgi_symbol", label = interesting_genes, size = 4, alpha = 1.0,
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]])

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p08_scn_volcano.pdf", width = 9, height = 9)
kh_p08_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_scn_volcano' not found

plotted <- dev.off()
kh_p08_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_scn_volcano' not found

## why in the crap is it double-labelling!?
## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p08_scn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p08_scn_ma.pdf", width = 9, height = 9)
kh_p08_scn_ma[["plot"]]

## Error:
## ! object 'kh_p08_scn_ma' not found

plotted <- dev.off()
kh_p08_scn_ma[["plot"]]

## Error:
## ! object 'kh_p08_scn_ma' not found

19.5.2.1 Same plot but a different set of labeled genes

table_name <- "kh_p08_scn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting_genes <- c(
  "Anapc16", "Gabra2", "Tmco1", "Sod2", "Fgf9", "Pdcd4", "Rhoa", "Gsk3b", "Foxp1",
  "Ncam1", "Marcks", "Fign", "Dpysl3", "Inpp5f", "Cadm1", "Map4", "Ugcg", "Elovl4",
  "Elavl1", "Cfl2", "Tnnt1", "Gnb1", "Impact", "Nrn1", "Nlgn3", "Actb", "Cd47",
  "Sec22b", "Slc17a7", "Vglut1", "Actb", "B4galt5", "Foxp1", "Otx2", "Lin7c",
  "Snx12", "Atg12", "Set")
kh_p08_scn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  label_column = "mgi_symbol", label = interesting_genes, size = 4, alpha = 1.0)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p08_scn_volcano_v2.pdf", width = 9, height = 9)
kh_p08_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_scn_volcano' not found

plotted <- dev.off()
kh_p08_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_scn_volcano' not found

## why in the crap is it double-labelling!?
## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p08_scn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p08_scn_ma_v2.pdf", width = 9, height = 9)
kh_p08_scn_ma[["plot"]]

## Error:
## ! object 'kh_p08_scn_ma' not found

plotted <- dev.off()
kh_p08_scn_ma[["plot"]]

## Error:
## ! object 'kh_p08_scn_ma' not found

19.5.2.2 ko/het p08 dLGN

table_name <- "kh_p08_dlgn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
kh_p08_dlgn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_dlgn"]], color_high = colors[["het_dlgn"]],
  label_column = "mgi_symbol", label = 10, size = 4, alpha = 1.0)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p08_dlgn_volcano.pdf", width = 9, height = 9)
kh_p08_dlgn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_dlgn_volcano' not found

plotted <- dev.off()
kh_p08_dlgn_volcano[["plot"]]

## Error:
## ! object 'kh_p08_dlgn_volcano' not found

## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p08_dlgn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_dlgn"]], color_high = colors[["het_dlgn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = 10)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p08_dlgn_ma.pdf", width = 9, height = 9)
kh_p08_dlgn_ma[["plot"]]

## Error:
## ! object 'kh_p08_dlgn_ma' not found

plotted <- dev.off()
kh_p08_dlgn_ma[["plot"]]

## Error:
## ! object 'kh_p08_dlgn_ma' not found

19.5.2.3 ko/het for p15 retinas

For some crazy reason, this plot is double-labelling!

table_name <- "kh_p15_retina"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting <- c("Opn4", "Gm9008", "Lrr1", "Cnbd1")
kh_p15_retina_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp", fill = "black",
  color_low = colors[["ko_retina"]], color_high = colors[["het_retina"]],
  label_column = "mgi_symbol", label = interesting, alpha = 1.0,
  size = 4)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p15_retina_volcano.pdf", width = 9, height = 9)
kh_p15_retina_volcano[["plot"]]

## Error:
## ! object 'kh_p15_retina_volcano' not found

plotted <- dev.off()
kh_p15_retina_volcano[["plot"]]

## Error:
## ! object 'kh_p15_retina_volcano' not found

## why in the crap is it double-labelling!?
## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p15_retina_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_retina"]], color_high = colors[["het_retina"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p15_retina_ma.pdf", width = 9, height = 9)
kh_p15_retina_ma[["plot"]]

## Error:
## ! object 'kh_p15_retina_ma' not found

plotted <- dev.off()
kh_p15_retina_ma[["plot"]]

## Error:
## ! object 'kh_p15_retina_ma' not found

19.5.2.4 ko/het p15 SCN

Holy crappers, this plot did not double label; oooh I have a check in my plotter to see if there are too few/too many labels and I foolishly allowed it to concatenate the labels! What in the crap was I thinking?

I am going to make an executive decision for this plot, 15 is too many and makes it crazy cluttered.

19.5.3 Repeat this with two sets of genes

table_name <- "kh_p15_scn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting_genes <- c("Fign", "Nrn1", "Dpysl2", "Actb", "Fgf9", "Otx2", "Sec23",
                       "Ncam1", "Map4", "Sec22b", "Nlgn3", "Marcks", "Cd47",
                       "Dpysl3", "Lin7c", "Cadm1", "Snx12", "Rhoa", "Inpp5f",
                       "Atg12", "Set", "Gsk3b", "Pdcd4", "Gabra2", "Tmco1", "Anapc16")
kh_p15_scn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  label_column = "mgi_symbol", size = 4, alpha = 1.0,
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]])

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p15_scn_volcano.pdf", width = 9, height = 9)
kh_p15_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_scn_volcano' not found

plotted <- dev.off()
kh_p15_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_scn_volcano' not found

## why in the crap is it double-labelling!?
## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p15_scn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p15_scn_ma.pdf", width = 9, height = 9)
kh_p15_scn_ma[["plot"]]

## Error:
## ! object 'kh_p15_scn_ma' not found

plotted <- dev.off()
kh_p15_scn_ma[["plot"]]

## Error:
## ! object 'kh_p15_scn_ma' not found

table_name <- "kh_p15_scn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
interesting_genes <- c(
  "Anapc16", "Gabra2", "Tmco1", "Sod2", "Fgf9", "Pdcd4", "Rhoa", "Gsk3b", "Foxp1",
  "Ncam1", "Marcks", "Fign", "Dpysl3", "Inpp5f", "Cadm1", "Map4", "Ugcg", "Elovl4",
  "Elavl1", "Cfl2", "Tnnt1", "Gnb1", "Impact", "Nrn1", "Nlgn3", "Actb", "Cd47",
  "Sec22b", "Slc17a7", "Vglut1", "Actb", "B4galt5", "Foxp1", "Otx2", "Lin7c",
  "Snx12", "Atg12", "Set")
kh_p15_scn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  label_column = "mgi_symbol", label = interesting_genes, size = 4, alpha = 1.0)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p15_scn_volcano_v2.pdf", width = 9, height = 9)
kh_p15_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_scn_volcano' not found

plotted <- dev.off()
kh_p15_scn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_scn_volcano' not found

## why in the crap is it double-labelling!?
## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p15_scn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["het_scn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p15_scn_ma_v2.pdf", width = 9, height = 9)
kh_p15_scn_ma[["plot"]]

## Error:
## ! object 'kh_p15_scn_ma' not found

plotted <- dev.off()
kh_p15_scn_ma[["plot"]]

## Error:
## ! object 'kh_p15_scn_ma' not found

19.5.3.1 ko/het p15 dLGN

table_name <- "kh_p15_dlgn"
table_input <- genotype_tables[[table_name]]
table <- table_input[["data"]][[table_name]]
kh_p15_dlgn_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_dlgn"]], color_high = colors[["het_dlgn"]],
  label_column = "mgi_symbol", label = 10, size = 4, alpha = 1.0)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/kh_p15_dlgn_volcano.pdf", width = 9, height = 9)
kh_p15_dlgn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_dlgn_volcano' not found

plotted <- dev.off()
kh_p15_dlgn_volcano[["plot"]]

## Error:
## ! object 'kh_p15_dlgn_volcano' not found

## My MA plotter isn't as smart as the volcano plotter, the genes are:
kh_p15_dlgn_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_dlgn"]], color_high = colors[["het_dlgn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = 10)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/kh_p15_dlgn_ma.pdf", width = 9, height = 9)
kh_p15_dlgn_ma[["plot"]]

## Error:
## ! object 'kh_p15_dlgn_ma' not found

plotted <- dev.off()
kh_p15_dlgn_ma[["plot"]]

## Error:
## ! object 'kh_p15_dlgn_ma' not found

19.5.4 Location contrasts with genes removed/kept

Repeat the same block with a find/replace of genotype/location.

location_tables_full <- combine_de_tables(
  location_de, keepers = location_keepers, label_column = label_column,
  excel = glue("full_contrasts/location_full_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage3118e0199102e92b3facb25dddf62f4b.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage7c02ab8ec90fd1f594de8f1d2a4d3272.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage8a6e79110110aca32ad01f58296b025f.png

location_tables_full

## A set of combined differential expression results.

##                             table deseq_sigup deseq_sigdown edger_sigup
## 1  p08_het_dlgn_vs_p08_het_retina           0             0           0
## 2  p15_het_dlgn_vs_p15_het_retina           0             0           0
## 3    p08_ko_dlgn_vs_p08_ko_retina           0             0           0
## 4    p15_ko_dlgn_vs_p15_ko_retina           0             0           0
## 5   p08_het_scn_vs_p08_het_retina           0             0           0
## 6   p15_het_scn_vs_p15_het_retina           0             0           0
## 7     p08_ko_scn_vs_p08_ko_retina           0             0           0
## 8     p15_ko_scn_vs_p15_ko_retina           0             0           0
## 9     p08_het_dlgn_vs_p08_het_scn           0             0           0
## 10    p15_het_dlgn_vs_p15_het_scn           0             0           0
## 11      p08_ko_dlgn_vs_p08_ko_scn           0             0           0
## 12      p15_ko_dlgn_vs_p15_ko_scn           0             0           0
##    edger_sigdown limma_sigup limma_sigdown
## 1              0        1997          1792
## 2              0        2970          2846
## 3              0        2297          2098
## 4              0        3289          3125
## 5              0        2146          1816
## 6              0        2686          2375
## 7              0        2372          2108
## 8              0        2799          2635
## 9              0         629           736
## 10             0        1984          1989
## 11             0        1088          1124
## 12             0        1879          1975

## Only  has information, cannot create an UpSet.

## Plot describing unique/shared genes in a differential expression table.

## NULL

location_sig_full <- extract_significant_genes(
  location_tables_full, according_to = "deseq",
  excel = glue("full_contrasts/location_full_sig-v{ver}.xlsx"))

## Error in `extract_significant_genes()`:
## ! object 'do_excel' not found

location_sig_full

## Error:
## ! object 'location_sig_full' not found

location_full_upset <- upsetr_sig(location_sig_full)

## Error:
## ! object 'location_sig_full' not found

##location_full_intersects <- write_upset_groups(
##  location_full_upset,
##  excel = "excel/location_full_gene_groups.xlsx")
location_tables <- list()
location_sig <- list()
location_gp <- list()
location_cp <- list()
for (k in seq_along(location_keepers)) {
  name <- names(location_keepers)[k]
  message("Examining ", name)
  keeper <- location_keepers[name]
  includes <- location_inclusions[[name]]
  include_name <- paste0("inc_", name)
  include_df_name <- paste0("df_", name)
  include_df <- location_inclusions[[include_df_name]]
  includes <- location_inclusions[[include_name]]
  summary(rownames(location_sig_full[["deseq"]][["ups"]][[name]]) %in% includes)
  include_filename <- glue("location_contrasts/location_{name}_including_wt_{lfc_cutoff}_decreased_table-v{ver}.xlsx")
  include_sig_filename <- glue("location_contrasts/location_{name}_including_wt_{lfc_cutoff}_decreased_sig-v{ver}.xlsx")
  location_tables[[name]] <- combine_de_tables(
    location_de, extra_annot = include_df,
    keepers = keeper, label_column = label_column,
    excel = include_filename, wanted_genes = includes)
  print(location_tables[[name]])
  location_sig[[name]] <- extract_significant_genes(
    location_tables[[name]], according_to = "deseq",
    excel = include_sig_filename)
  print(location_sig[[name]])
  num_rows <- nrow(location_sig[[name]][["deseq"]][["ups"]][[name]]) +
    nrow(location_sig[[name]][["deseq"]][["downs"]][[name]])
  message("There are ", num_rows, " significant up and down genes.")
  if (num_rows > 10) {
    location_gp[[name]] <- all_gprofiler(location_sig[[name]], species = "mmusculus")
    gp_written <- write_all_gp(genotype_gp[[name]])
    location_cp[[name]] <- all_cprofiler(location_sig[[name]], location_tables[[name]],
                                         orgdb = "org.Mm.eg.db")
    cp_written <- write_all_cp(genotype_cp[[name]])
  }
}

## Examining dr_p08_het

## Error:
## ! object 'location_sig_full' not found

Colenso sent a specific query of interest, comparing SCN vs. Retinas at p08 in the heterozygotes including a set of genes of particular interest. Perhaps I can use some of these as markers to quality control my work in the future?

Here are the genes:

Opn4, Eomes, Trpc7, Oprm1, Nr4a3, Tbx20, Irx6, AW551984, Pcdh19, Adcyap1, Baiap3, Chl1, Grin3a, Igf1, Gria1, Grin2d, Grin3a, Chrna6, Chrna3, Htr5a, Htr2a, Htr7, Irx4, PlxnC1, Sema6d, Sema4f, Sema4a, Sema6b, Lrrc4b, Lrrc58, Lrrc3b, Wnt4, Wnt9b, Ctxn3, Tenm1, Gna14, Rgs4, Rgs6, Rgs5

table_input <- location_tables[["sr_p08_het"]]
table_name <- "sr_p08_het"
table <- table_input[["data"]][[table_name]]
interesting_genes <- c("Opn4", "Eomes", "Trpc7", "Oprm1", "Nr4a3", "Tbx20",
                       "Irx6", "AW551984", "Pcdh19", "Adcyap1r1", "Baiap3",
                       "Chl1", "Grin3a", "Igf1", "Gria1", "Grin2d", "Grin3a",
                       "Chrna6", "Chrna3", "Htr5a", "Htr2a", "Htr7", "Irx4",
                       "PlxnC1", "Sema6d", "Sema4f", "Sema4a", "Sema6b", "Lrrc4b",
                       "Lrrc58", "Lrrc3b", "Wnt4", "Wnt9b", "Ctxn3", "Tenm1", "Gna14",
                       "Rgs4", "Rgs6", "Rgs5", "Pou4f2", "Chrnb3", "Bcan")
sr_p08_het_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["het_scn"]], color_high = colors[["het_retina"]],
  label_column = "mgi_symbol", label = interesting_genes, alpha = 1.0,
  size = 4)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/sr_p08_het_volcano.pdf", width = 9, height = 9)
sr_p08_het_volcano[["plot"]]

## Error:
## ! object 'sr_p08_het_volcano' not found

plotted <- dev.off()
sr_p08_het_volcano[["plot"]]

## Error:
## ! object 'sr_p08_het_volcano' not found

sr_p08_het_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["het_scn"]], color_high = colors[["het_retina"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/sr_p08_het_ma.pdf", width = 9, height = 9)
sr_p08_het_ma[["plot"]]

## Error:
## ! object 'sr_p08_het_ma' not found

plotted <- dev.off()
sr_p08_het_ma[["plot"]]

## Error:
## ! object 'sr_p08_het_ma' not found

19.5.5 Also the ko

table_input <- location_tables[["sr_p08_ko"]]
table_name <- "sr_p08_ko"
table <- table_input[["data"]][[table_name]]
sr_p08_ko_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_scn"]], color_high = colors[["ko_retina"]],
  label_column = "mgi_symbol", label = interesting_genes, alpha = 1.0,
  size = 4)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/sr_p08_ko_volcano.pdf", width = 9, height = 9)
sr_p08_ko_volcano[["plot"]]

## Error:
## ! object 'sr_p08_ko_volcano' not found

plotted <- dev.off()
sr_p08_ko_volcano[["plot"]]

## Error:
## ! object 'sr_p08_ko_volcano' not found

sr_p08_ko_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["ko_retina"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/sr_p08_ko_ma.pdf", width = 9, height = 9)
sr_p08_ko_ma[["plot"]]

## Error:
## ! object 'sr_p08_ko_ma' not found

plotted <- dev.off()
sr_p08_ko_ma[["plot"]]

## Error:
## ! object 'sr_p08_ko_ma' not found

table_input <- location_tables[["sr_p15_het"]]
table_name <- "sr_p15_het"
table <- table_input[["data"]][[table_name]]
interesting_genes <- c("Opn4", "Eomes", "Trpc7", "Oprm1", "Nr4a3", "Tbx20",
                       "Irx6", "AW551984", "Pcdh19", "Adcyap1r1", "Baiap3",
                       "Chl1", "Grin3a", "Igf1", "Gria1", "Grin2d", "Grin3a",
                       "Chrna6", "Chrna3", "Htr5a", "Htr2a", "Htr7", "Irx4",
                       "PlxnC1", "Sema6d", "Sema4f", "Sema4a", "Sema6b", "Lrrc4b",
                       "Lrrc58", "Lrrc3b", "Wnt4", "Wnt9b", "Ctxn3", "Tenm1", "Gna14",
                       "Rgs4", "Rgs6", "Rgs5", "Pou4f2", "Chrnb3", "Bcan")
sr_p15_het_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["het_scn"]], color_high = colors[["het_retina"]],
  label_column = "mgi_symbol", label = interesting_genes, alpha = 1.0,
  size = 4)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/sr_p15_het_volcano.pdf", width = 9, height = 9)
sr_p15_het_volcano[["plot"]]

## Error:
## ! object 'sr_p15_het_volcano' not found

plotted <- dev.off()
sr_p15_het_volcano[["plot"]]

## Error:
## ! object 'sr_p15_het_volcano' not found

sr_p15_het_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["het_retina"]], color_high = colors[["het_scn"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/sr_p15_het_ma.pdf", width = 9, height = 9)
sr_p15_het_ma[["plot"]]

## Error:
## ! object 'sr_p15_het_ma' not found

plotted <- dev.off()
sr_p15_het_ma[["plot"]]

## Error:
## ! object 'sr_p15_het_ma' not found

19.5.6 Also the ko

table_input <- location_tables[["sr_p15_ko"]]
table_name <- "sr_p15_ko"
table <- table_input[["data"]][[table_name]]
sr_p15_ko_volcano <- plot_volcano_condition_de(
  table, table_name, fc_col = "deseq_logfc", p_col = "deseq_adjp",
  color_low = colors[["ko_retina"]], color_high = colors[["ko_scn"]],
  label_column = "mgi_symbol", label = interesting_genes, alpha = 1.0,
  size = 4, min.segment.length = 0, point.padding = 0.2)

## Error in `plot_volcano_condition_de()`:
## ! Column: deseq_logfc is not in the table.

pp(file = "images/sr_p15_ko_volcano.pdf", width = 12, height = 12)
sr_p15_ko_volcano[["plot"]]

## Error:
## ! object 'sr_p15_ko_volcano' not found

plotted <- dev.off()
sr_p15_ko_volcano[["plot"]]

## Error:
## ! object 'sr_p15_ko_volcano' not found

sr_p15_ko_ma <- plot_ma_condition_de(
  table, table_name, expr_col = "deseq_basemean", fc_col = "deseq_logfc",
  color_low = colors[["ko_scn"]], color_high = colors[["ko_retina"]],
  p_col = "deseq_adjp", label_column = "mgi_symbol", label = interesting_genes)

## The column: mgi_symbol is not in the data, using rownames.

## Warning in max(newdf[["avg"]]): no non-missing arguments to max; returning -Inf

## Warning in plot_ma_condition_de(table, table_name, expr_col = "deseq_basemean",
## : NAs introduced by coercion

## Error in `[[<-.data.frame`:
## ! replacement has 1 row, data has 0

pp(file = "images/sr_p15_ko_ma.pdf", width = 9, height = 9)
sr_p15_ko_ma[["plot"]]

## Error:
## ! object 'sr_p15_ko_ma' not found

plotted <- dev.off()
sr_p15_ko_ma[["plot"]]

## Error:
## ! object 'sr_p15_ko_ma' not found

19.5.6.1 Test a specific location query for duplicated IDs

Let us see if any Ensembl gene IDs and/or MGI IDs are shared in the worksheet location_sr_p08_ko_including_wt_0.1_decreased_sig up/down.

test_table_up <- location_sig[["sr_p08_ko"]][["deseq"]][["ups"]][[1]]
test_table_down <- location_sig[["sr_p08_ko"]][["deseq"]][["downs"]][[1]]

query <- list("up" = rownames(test_table_up),
              "down" = rownames(test_table_down))
query_upset <- UpSetR::fromList(query)
UpSetR::upset(query_upset)

## Error in `start_col:end_col`:
## ! argument of length 0

query <- list("up" = test_table_up[["mgi_symbol"]],
              "down" = test_table_down[["mgi_symbol"]])
query_upset <- UpSetR::fromList(query)
UpSetR::upset(query_upset)

## Error in `start_col:end_col`:
## ! argument of length 0

## ok, good.

19.5.7 And time

time_tables_full <- combine_de_tables(
  time_de, keepers = time_keepers,
  label_column = label_column,
  excel = glue("full_contrasts/time_full_tables-v{ver}.xlsx"))

## Looking for subscript invalid names, start of extract_keepers.

## Looking for subscript invalid names, end of extract_keepers.

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage44a7ee65d962a1c210efb423acb91f9f.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage5573f66508b27264d423d154c596f319.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImagea2a06e763ff9a48367230346bae980b4.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage93e639ffc275dda6b527518d63e7e518.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage18e95ba34cbcde6c5c14f932ca13f04c.png

## Error in compute.Venn(V, doWeights = doWeights, doEuler = doEuler, type = type) : 
##   Not enough sets
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = tmp_file) : too many open devices
## Error in png(filename = png_name, width = width, height = height, units = units,  : 
##   too many open devices

## The png file name did not exist: /lab/singularity/iprgcv2/202602261159_outputs/tmp/RtmpoB5f96/figureImage5cd44d9aba389dbd82bfb6640acf1a9d.png

time_sig_full <- extract_significant_genes(
  time_tables_full, according_to = "deseq",
  excel = glue("full_contrasts/time_full_sig-v{ver}.xlsx"))

## Error in `extract_significant_genes()`:
## ! object 'do_excel' not found

time_tables <- list()
time_sig <- list()
time_gp <- list()
time_cp <- list()
for (k in seq_along(time_keepers)) {
  name <- names(time_keepers)[k]
  message("Examining ", name)
  keeper <- time_keepers[name]
  includes <- time_inclusions[[name]]
  include_name <- paste0("inc_", name)
  include_df_name <- paste0("df_", name)
  include_df <- time_inclusions[[include_df_name]]
  includes <- time_inclusions[[include_name]]
  summary(rownames(time_sig_full[["deseq"]][["ups"]][[name]]) %in% includes)
  include_filename <- glue("time_contrasts/time_{name}_including_wt_{lfc_cutoff}_decreased_table-v{ver}.xlsx")
  include_sig_filename <- glue("time_contrasts/time_{name}_including_wt_{lfc_cutoff}_decreased_sig-v{ver}.xlsx")
  time_tables[[name]] <- combine_de_tables(
    time_de, extra_annot = include_df,
    keepers = keeper, label_column = label_column,
    excel = include_filename, wanted_genes = includes)
  print(time_tables[[name]])
  time_sig[[name]] <- extract_significant_genes(
    time_tables[[name]], according_to = "deseq",
    excel = include_filename)
  print(time_sig[[name]])
  num_rows <- nrow(time_sig[[name]][["deseq"]][["ups"]][[name]]) +
    nrow(time_sig[[name]][["deseq"]][["downs"]][[name]])
  message("There are ", num_rows, " significant up and down genes.")
  if (num_rows > 10) {
    time_gp[[name]] <- all_gprofiler(time_sig[[name]], species = "mmusculus")
    gp_written <- write_all_gp(time_gp[[name]])
    time_cp[[name]] <- all_cprofiler(time_sig[[name]], time_tables[[name]],
                                     orgdb = "org.Mm.eg.db")
    cp_written <- write_all_cp(time_cp[[name]])
  }
}

## Examining t_het_dlgn

## Error:
## ! object 'time_sig_full' not found

20 Translatome queries

In conversation with Colenso, he spoke about a series of contrasts which would be interesting to attempt in order to query the changes across both locations and genotypes and/or both locations and time, thus:

(p08_het_scn / p08_het_retina) / (p08_ko_scn / p08_ko_retina)

as an example. We can definitely do these, but they do not work for all methods employed (I think they work best with limma and edgeR).

Lets find out!

20.1 Two scn/retina comparisons

• (p08_het_scn / p08_het_retina) / (p08_ko_scn / p08_ko_retina)
• (p15_het_scn / p15_het_retina) / (p15_ko_scn / p15_ko_retina)

scn_extra <- glue("\\
  p08het = (conditionp08_het_scn - conditionp08_het_retina), \\
  p08ko = (conditionp08_ko_scn - conditionp08_ko_retina), \\
  p08het_vs_p08ko = (conditionp08_het_scn - conditionp08_het_retina) - (conditionp08_ko_scn - conditionp08_ko_retina), \\
  p15het = (conditionp15_het_scn - conditionp15_het_retina), \\
  p15ko = (conditionp15_ko_scn - conditionp15_ko_retina), \\
  p15het_vs_p15ko = (conditionp15_het_scn - conditionp15_het_retina) - (conditionp15_ko_scn - conditionp15_ko_retina)")
scn_translatome_de_keepers <- list(
  "p08het" = c("p08_het_scn", "p08_het_retina"),
  "p08ko" = c("p08_ko_scn", "p08_ko_retina"),
  "p15het" = c("p15_het_scn", "p15_het_retina"),
  "p15ko" = c("p15_ko_scn", "p15_ko_retina"))
scn_translatome_keepers <- list(
  "p08het" = c("p08_het_scn", "p08_het_retina"),
  "p08ko" = c("p08_ko_scn", "p08_ko_retina"),
  "p08_scn_translatome" = c("p08het", "p08ko"),
  "p15het" = c("p15_het_scn", "p15_het_retina"),
  "p15ko" = c("p15_ko_scn", "p15_ko_retina"),
  "p15_scn_translatome" = c("p15het", "p15ko"))
filt <- normalize(v3_pairwise_input, filter = TRUE)

## Removing 10162 low-count genes (15263 remaining).

limma_test <- limma_pairwise(filt,
                             keepers = scn_translatome_de_keepers,
                             keep_underscore = TRUE,
                             model_batch = FALSE, extra_contrastrs = scn_extra)

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

edger_test <- edger_pairwise(filt,
                             keepers = scn_translatome_de_keepers,
                             keep_underscore = TRUE,
                             model_batch = FALSE, extra_contrasts = scn_extra)

## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(filt, keepers = scn_translatome_de_keepers,
## keep_underscore = TRUE, : estimateGLMCommonDisp() failed.  Trying again with
## estimateDisp().

## Warning in edger_pairwise(filt, keepers = scn_translatome_de_keepers,
## keep_underscore = TRUE, : There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in `glmFit.default()`:
## ! Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

scn_translatome_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                   keepers = scn_translatome_de_keepers,
                                   model_batch = FALSE,
                                   do_basic = FALSE, do_dream = FALSE,
                                   do_noiseq = FALSE, do_ebseq = FALSE,
                                   extra_contrasts = scn_extra,
                                   parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## scn_translatome_de_keepers, : This will likely fail because of how the keepers
## and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().

## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

scn_combined_test <- combine_de_tables(
  scn_translatome_de, keepers = scn_translatome_keepers,
  excel = glue("translatome/test_scn_translatome_unfiltered_nosva-v{ver}.xlsx"))

## Error:
## ! object 'scn_translatome_de' not found

scn_translatome_de_sva <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                       keepers = scn_translatome_de_keepers,
                                       model_batch = "svaseq",
                                       do_basic = FALSE, do_dream = FALSE,
                                       do_noiseq = FALSE, do_ebseq = FALSE,
                                       extra_contrasts = scn_extra,
                                       parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## scn_translatome_de_keepers, : This will likely fail because of how the keepers
## and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).
## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().
## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

scn_combined_test_sva <- combine_de_tables(
  scn_translatome_de_sva, keepers = scn_translatome_keepers,
  excel = glue("translatome/test_scn_translatome_unfiltered_sva-v{ver}.xlsx"))

## Error:
## ! object 'scn_translatome_de_sva' not found

20.1.1 Subtracting DESeq2 results: p08 scn het vs ko

p08_scn_combined_deseq <- subtract_deseq_results(
  first_table = scn_combined_test[["data"]][["p08het"]],
  second_table = scn_combined_test[["data"]][["p08ko"]],
  first_lfc = "deseq_logfc", second_lfc = "deseq_logfc",
  first_p = "deseq_adjp", second_p = "deseq_adjp",
  first_name = "het", second_name = "ko",
  excel = glue("translatome/translatome_p08_scn_combined_deseq-v{ver}.xlsx"))

## Error in `subtract_deseq_results()`:
## ! could not find function "subtract_deseq_results"

20.1.2 Subtracting DESeq2 results: p15 scn het vs ko

p15_scn_combined_deseq <- subtract_deseq_results(
  first_table = scn_combined_test[["data"]][["p15het"]],
  second_table = scn_combined_test[["data"]][["p15ko"]],
  first_lfc = "deseq_logfc", second_lfc = "deseq_logfc",
  first_p = "deseq_adjp", second_p = "deseq_adjp",
  first_name = "het", second_name = "ko",
  excel = glue("translatome/translatome_p15_scn_combined_deseq-v{ver}.xlsx"))

## Error in `subtract_deseq_results()`:
## ! could not find function "subtract_deseq_results"

20.2 One dlgn/retina comparison

• (p08_het_dlgn / p08_het_retina) / (p08_ko_dlgn / p08_ko_retina)

p08_dlgn_extra <- "p08het_vs_p08ko = (conditionp08_het_dlgn - conditionp08_het_retina) - (conditionp08_ko_dlgn - conditionp08_ko_retina)"
p08_dlgn_translatome_de_keepers <- list(
  "p08het" = c("p08_het_dlgn", "p08_het_retina"),
  "p08ko" = c("p08_ko_dlgn", "p08_ko_retina"))
p08_dlgn_translatome_keepers <- list(
  "p08_het_dlgn_vs_retina" = c("p08_het_dlgn", "p08_het_retina"),
  "p08_ko_dlgn_vs_retina" = c("p08_ko_dlgn", "p08_ko_retina"),
  "p08_dlgn_translatome" = c("p08het", "p08ko"))
p08_dlgn_translatome_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                        keepers = p08_dlgn_translatome_de_keepers,
                                        model_batch = FALSE,
                                        do_basic = FALSE, do_dream = FALSE,
                                        do_noiseq = FALSE, do_ebseq = FALSE,
                                        extra_contrasts = p08_dlgn_extra,
                                        parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## p08_dlgn_translatome_de_keepers, : This will likely fail because of how the
## keepers and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().

## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

p08_dlgn_combined_test <- combine_de_tables(
  p08_dlgn_translatome_de, keepers = p08_dlgn_translatome_keepers,
  label_column = label_column,
  excel = glue("translatome/test_p08_dlgn_translatome_unfiltered_nosva-v{ver}.xlsx"))

## Error:
## ! object 'p08_dlgn_translatome_de' not found

p08_dlgn_translatome_de_sva <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                            keepers = p08_dlgn_translatome_de_keepers,
                                            model_batch = "svaseq",
                                            do_basic = FALSE, do_dream = FALSE,
                                            do_noiseq = FALSE, do_ebseq = FALSE,
                                            extra_contrasts = p08_dlgn_extra,
                                            parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## p08_dlgn_translatome_de_keepers, : This will likely fail because of how the
## keepers and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).
## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().
## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

p08_dlgn_combined_test_sva <- combine_de_tables(
  p08_dlgn_translatome_de_sva, keepers = p08_dlgn_translatome_keepers,
  label_column = label_column,
  excel = glue("translatome/test_p08_dlgn_translatome_unfiltered_sva-v{ver}.xlsx"))

## Error:
## ! object 'p08_dlgn_translatome_de_sva' not found

20.2.1 Subtracting the DESeq2 results

20.3 Two scn/retina comparisons (p15/p08 across het/ko)

• (p15_het_scn / p15_het_retina) / (p08_het_scn / p08_het_retina)
• (p15_ko_scn / p15_ko_retina) / (p08_ko_scn / p08_ko_retina)

time_scn_extra <- glue("\\
  p15het = (conditionp15_het_scn - conditionp15_het_retina), \\
  p08het = (conditionp08_het_scn - conditionp08_het_retina), \\
  p15het_vs_p08het = (conditionp15_het_scn - conditionp15_het_retina) - (conditionp08_het_scn - conditionp08_het_retina),
  p15ko = (conditionp15_ko_scn - conditionp15_ko_retina), \\
  p08ko = (conditionp08_ko_scn - conditionp08_ko_retina), \\
  p15ko_vs_p08ko = (conditionp15_ko_scn - conditionp15_ko_retina) - (conditionp08_ko_scn - conditionp08_ko_retina)")
time_scn_translatome_de_keepers <- list(
  "p15het" = c("p15_het_scn", "p15_het_retina"),
  "p08het" = c("p08_het_scn", "p08_het_retina"),
  "p15ko" = c("p15_ko_scn", "p15_ko_retina"),
  "p08ko" = c("p08_ko_scn", "p08_ko_retina"))
time_scn_translatome_keepers <- list(
  "p15het" = c("p15_het_scn", "p15_het_retina"),
  "p08het" = c("p08_het_scn", "p08_het_retina"),
  "p15ko" = c("p15_ko_scn", "p15_ko_retina"),
  "p08ko" = c("p08_ko_scn", "p08_ko_retina"),
  "p15_het_sc_vs_retina" = c("p15_het_scn", "p15_het_retina"),
  "p08_het_sc_vs_retina" = c("p08_het_scn", "p08_het_retina"),
  "scn_het_translatome" = c("p15het", "p08het"),
  "scn_ko_translatome" = c("p15ko", "p08ko"))
time_scn_translatome_de <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                        keepers = time_scn_translatome_de_keepers,
                                        model_batch = FALSE,
                                        do_basic = FALSE, do_dream = FALSE,
                                        do_noiseq = FALSE, do_ebseq = FALSE,
                                        extra_contrasts = time_scn_extra,
                                        parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## time_scn_translatome_de_keepers, : This will likely fail because of how the
## keepers and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).

## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().

## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

time_scn_translatome_test <- combine_de_tables(
  time_scn_translatome_de,
  keepers = time_scn_translatome_keepers,
  label_column = label_column,
  excel = glue("translatome/test_time_scn_translatome_unfiltered_nosva-v{ver}.xlsx"))

## Error:
## ! object 'time_scn_translatome_de' not found

time_scn_translatome_de_sva <- all_pairwise(v3_pairwise_input, filter = TRUE,
                                            keepers = time_scn_translatome_de_keepers,
                                            model_batch = "svaseq",
                                            do_basic = FALSE, do_dream = FALSE,
                                            do_noiseq = FALSE, do_ebseq = FALSE,
                                            extra_contrasts = time_scn_extra,
                                            parallel = FALSE, keep_underscore = TRUE)

## Warning in all_pairwise(v3_pairwise_input, filter = TRUE, keepers =
## time_scn_translatome_de_keepers, : This will likely fail because of how the
## keepers and extra contrasts are evaluated.

##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## p08 p15 
##  31  32

## Warning: attributes are not identical across measure variables; they will be
## dropped

## Removing 10162 low-count genes (15263 remaining).
## converting counts to integer mode

## Error in checkFullRank(modelMatrix) : 
##   the model matrix is not full rank, so the model cannot be fit as specified.
##   One or more variables or interaction terms in the design formula are linear
##   combinations of the others and must be removed.
## 
##   Please read the vignette section 'Model matrix not full rank':
## 
##   vignette('DESeq2')
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Error in glmFit.default(y, design = design, dispersion = dispersion, offset = offset,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15

## Warning in edger_pairwise(...): estimateGLMCommonDisp() failed.  Trying again
## with estimateDisp().
## Warning in edger_pairwise(...): There was a failure when doing the estimations.

## There was a failure when doing the estimations, using estimateDisp().

## Error in glmFit.default(sely, design, offset = seloffset, dispersion = 0.05,  : 
##   Design matrix not of full rank.  The following coefficients not estimable:
##  batchp15
## conditions
##   p08_het_dlgn p08_het_retina    p08_het_scn    p08_ko_dlgn  p08_ko_retina 
##              3              3              3              3              3 
##     p08_ko_scn    p08_wt_dlgn  p08_wt_retina     p08_wt_scn   p15_het_dlgn 
##              3              5              5              3              4 
## p15_het_retina    p15_het_scn    p15_ko_dlgn  p15_ko_retina     p15_ko_scn 
##              4              3              3              3              3 
##    p15_wt_dlgn  p15_wt_retina     p15_wt_scn 
##              5              5              2 
## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Coefficients not estimable: batchp15

## Warning: Partial NA coefficients for 15263 probe(s)

## Error in `retlst[[meth]]`:
## ! attempt to select less than one element in get1index

time_scn_translatome_test_sva <- combine_de_tables(
  time_scn_translatome_de_sva,
  keepers = time_scn_translatome_keepers,
  label_column = label_column,
  excel = glue("translatome/test_time_scn_translatome_unfiltered_sva-v{ver}.xlsx"))

## Error:
## ! object 'time_scn_translatome_de_sva' not found

Next step: Perform the retina filter; need to think about the proper union/intersection of the retina/x expression values

In the previous block, we are making 2 global comparisons, here is one of them:

(p15hetscn/p15hetret)/(p08hetscn/p08hetret)

I therefore want to extract the most logical set of genes higher in some/all of these conditions with respect to the corresponding wt conditions. Previously, in section ‘Extract genes included for each set of contrasts’, I attempted to perform this operation for 2 specific wt conditions. When this was performed, it took the unique(union) of the two sets. Thus it stands to reason that I want to take the unique(union) of all 4 in this instance? e.g.:

(p15hetscn > p15wtscn) | (p15hetret > p15wtret) | (p08hetscn > p08wtscn) | (p08hetret > p08wtret)

I kind of think it should be:

((p15hetscn > p15wtscn) | (p15hetret > p15wtret)) & ((p08hetscn > p08wtscn) | (p08hetret > p08wtret))

gross, perhaps I should just do this manually, given that there are only a few putative translatomes to query?

21 Quick and dirty DESeq2 contrast of contrasts

In a fashion similar to how Hector handled the effect of phagocytosis with Laura and Najib a long time ago, I propose to do a simple subtraction of the results of our two contrasts which comprise the translatome query (I was thinking about this last week, thus the inclusion of them in the de tables above). Similarly to the phagocytosis effect, I will simply take the worst posible adjusted p-value. I will repeat this with limma/EdgeR and see how similar the final results are to what those methods provide in the (a/b)/(c/d) comparisons. I am reasonably certain that DESeq2’s results() function has the ability to perform these odd contrasts, but I have never figured out how; perhaps I will use this as a chance to revisit that…

Let us test this idea with the p08 dlgn query, which seeks to compare:

(p08_het_dlgn / p08_het_retina) / (p08_ko_dlgn / p08_ko_retina)

These are maintained in the de_table with the names ‘p08_het_dlgn_vs_retina’ and ‘p08_ko_dlgn_vs_retina’

21.1 p08 dlgn het vs ko

p08_dlgn_combined_deseq <- subtract_deseq_results(
  first_table = p08_dlgn_combined_test[["data"]][["p08_het_dlgn_vs_retina"]],
  second_table = p08_dlgn_combined_test[["data"]][["p08_ko_dlgn_vs_retina"]],
  first_lfc = "deseq_logfc", second_lfc = "deseq_logfc",
  first_p = "deseq_adjp", second_p = "deseq_adjp",
  first_name = "het", second_name = "ko",
  excel = glue("translatome/translatome_p08_dlgn_combined_deseq-v{ver}.xlsx"))

## Error in `subtract_deseq_results()`:
## ! could not find function "subtract_deseq_results"

See how similar these results are to those obtained from limma/edger.

test_columns <- c("edger_logfc", "limma_logfc", "edger_adjp", "limma_adjp")
test_df <- p08_dlgn_combined_test[["data"]][["p08_dlgn_translatome"]][, test_columns]

## Error:
## ! object 'p08_dlgn_combined_test' not found

test_df <- merge(test_df, p08_dlgn_combined_deseq, by = "row.names")

## Error:
## ! object 'test_df' not found

rownames(test_df) <- test_df[["Row.names"]]

## Error:
## ! object 'test_df' not found

test_df[["Row.names"]] <- NULL

## Error:
## ! object 'test_df' not found

cor.test(test_df[["limma_logfc"]], test_df[["het_vs_ko_logfc"]])

## Error:
## ! object 'test_df' not found

cor.test(test_df[["edger_logfc"]], test_df[["het_vs_ko_logfc"]])

## Error:
## ! object 'test_df' not found

tt <- plot_linear_scatter(test_df[, c("limma_logfc", "het_vs_ko_logfc")])

## Error:
## ! object 'test_df' not found

tt[["scatter"]]

## NULL

tt <- plot_linear_scatter(test_df[, c("edger_adjp", "het_vs_ko_p")])

## Error:
## ! object 'test_df' not found

tt[["scatter"]]

## NULL

## So, using the maximum p-value is a complete failure; but the extreme similarities
## between this and edgeR suggest to me that it is likely possible to use the results
## from edgeR without concern (or limma for that matter, it was also extremely similar)
## Or I can spend a little time and collect the numbers on each side of the division
## and calculate a t statistic myself.

22 Non-Specific filtering of the translatome data

I have on hand

• p08het_vs_p08ko : (p08_het_scn - p08_het_retina) - (p08_ko_scn - p08_ko_retina)
• p15het_vs_p15ko : (p15_het_scn - p15_het_retina) - (p15_ko_scn - p15_ko_retina)
• p08het_vs_p08ko : (p08_het_dlgn - p08_het_retina) - (p08_ko_dlgn - p08_ko_retina)
• p15het_vs_p08het : (p15_het_scn - p15_het_retina) - (p08_het_scn - p08_het_retina)
• p15ko_vs_p08ko : (p15_ko_scn - p15_ko_retina) - (p08_ko_scn - p08_ko_retina)

I have gene sets up above which define the genes suitable for each of these pieces. There are only 5 comparisons, let us step through them.

22.1 SCN translatome het/ko at p08

The data for this contrast resides in scn_combined_test\(data\)p08_scn_translatome or the same slot of scn_combined_test_sva

• p08_het_scn - p08_het_retina) - (p08_ko_scn - p08_ko_retina)

Thus, the inclusion_sig portions to extract are found in: inclusion_sig[[“deseq”]][[“ups”]], and are named exactly as written above!

p08_het_vs_ko_translatome_unfilt <- scn_combined_test[["data"]][["p08_scn_translatome"]]

## Error:
## ! object 'scn_combined_test' not found

num_union <- unique(c(rownames(inclusion_sig[["deseq"]][["ups"]][["p08_het_scn"]]),
                      rownames(inclusion_sig[["deseq"]][["ups"]][["p08_het_retina"]])))
length(num_union)

## [1] 1188

den_union <- unique(c(rownames(inclusion_sig[["deseq"]][["ups"]][["p08_ko_scn"]]),
                      rownames(inclusion_sig[["deseq"]][["ups"]][["p08_ko_retina"]])))
length(den_union)

## [1] 1624

both_union <- unique(c(num_union, den_union))
length(both_union)

## [1] 2005

both_inter_idx <- num_union %in% den_union
both_inter <- num_union[both_inter_idx]
length(both_inter)

## [1] 807

keeper <- list("p08_scn_translatome" = c("p08het", "p08ko"))
p08_scn_translatome_union_filtered <- combine_de_tables(
  scn_translatome_de, keepers = keeper,
  label_column = label_column,
  excel = glue("translatome/p08_scn_translatome_union_filtered_nosva-v{ver}.xlsx"),
  wanted_genes = both_union)

## Error:
## ! object 'scn_translatome_de' not found

p08_scn_translatome_inter_filtered <- combine_de_tables(
  scn_translatome_de, keepers = keeper,
  label_column = label_column,
  excel = glue("translatome/p08_scn_translatome_intersect_filtered_nosva-v{ver}.xlsx"),
  wanted_genes = both_inter)

## Error:
## ! object 'scn_translatome_de' not found

p08_scn_translatome_union_filtered_sva <- combine_de_tables(
  scn_translatome_de_sva, keepers = keeper,
  label_column = label_column,
  excel = glue("translatome/p08_scn_translatome_union_filtered_sva-v{ver}.xlsx"),
  wanted_genes = both_union)

## Error:
## ! object 'scn_translatome_de_sva' not found

p08_scn_translatome_union_filtered <- combine_de_tables(
  scn_translatome_de, keepers = keeper,
  label_column = label_column,
  excel = glue("translatome/p08_scn_translatome_intersect_filtered_sva-v{ver}.xlsx"),
  wanted_genes = both_inter)

## Error:
## ! object 'scn_translatome_de' not found

23 Venn/UpSet of Retina, SCN, and dLGN DE Genes

Here is a snippet from Rashmi which expresses nicely the DE-result comparisons she is most interested:

Since, I want to know the number of DEG expressed in Retina, SCN and dLGN with respect to genotype, Location and time. I prepared the venn diagram for these comparison:

• Genotype: P8 Ret Het vs KO, P15 Ret Het vs KO, P8 SCN Het vs KO, P15 SCN Het vs KO, P8 dLGN Het vs KO, P15 dLGN Het vs KO
• Location: P8_het Ret vs SCN, P8_KO Ret vs SCN, P15_het Ret vs SCN, P15_KO Ret vs SCN, P8_het Ret vs dLGN, P8_KO Ret vs dLGN, P15_het Ret vs dLGN, P15_KO Ret vs dLGN, P8_het SCN vs dLGN, P8_KO SCN vs dLGN, P15_het SCN vs dLGN, P15_KO SCN vs dLGN.

Since I was interested in understanding the change in local translatome according to Location for different developmental time points for Het and KO. Hence, I tried to generate a venn diagram for Location (Ret and SCN) at developmental time points P8 and P15 for genotype het and KO. So the venn diagram / upset plot will be for location where some genes will be shared/unique for P8_Ret_het, P8_SCN_Het, P15_Ret_HET, P15_SCN_HET. We can prepare an upset plot for P8_Ret_KO, P8_SCN_KO, P15_Ret_KO and P15_SCN_KO also. Or can generate an upset plot by combining both P8_Ret_het, P8_SCN_Het, P15_Ret_HET and P15_SCN_HET and P8_Ret_KO, P8_SCN_KO, P15_Ret_KO and P15_SCN_KO.

Ok, let us see if I can implement this, starting with the genotype query

• Genotype: P8 Ret Het vs KO, P15 Ret Het vs KO, P8 SCN Het vs KO, P15 SCN Het vs KO, P8 dLGN Het vs KO, P15 dLGN Het vs KO

23.1 ko vs het; all locations and times

## The appropriate data structure is 'genotype_tables',
## and the tables of interest are:
table_names <- c("kh_p08_retina", "kh_p15_retina", "kh_p08_scn",
                 "kh_p15_scn", "kh_p08_dlgn", "kh_p15_dlgn")
table_names %in% names(genotype_sig)

## [1] FALSE FALSE FALSE FALSE FALSE FALSE

newsig <- genotype_sig[[1]]

## Error in `genotype_sig[[1]]`:
## ! subscript out of bounds

for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- genotype_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- genotype_sig[[name]][["deseq"]][["downs"]][[name]]
}

## Error:
## ! object 'newsig' not found

genotype_upsetr <- upsetr_sig(newsig)

## Error:
## ! object 'newsig' not found

genotype_upset_written <- write_upset_groups(genotype_upsetr, excel = "excel/genotype_upset_groups.xlsx")

## Error:
## ! object 'genotype_upsetr' not found

genotype_upsetr[["all_plot"]]

## Error:
## ! object 'genotype_upsetr' not found

pp(file = "images/test_genotype_upset.pdf")
print(genotype_upsetr[["all_plot"]])

## Error:
## ! object 'genotype_upsetr' not found

plotted <- dev.off()

Now let us try the location-specific comparisons

23.2 scn vs retina, p08

## The appropriate data structure is 'genotype_tables',
## and the tables of interest are:
table_names <- c("sr_p08_het", "sr_p08_ko")
table_names %in% names(location_sig)

## [1] FALSE FALSE

location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/sr_p08_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

pp(file = "images/test_location_sr_p08_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

I am reasonably certain that Rashmi would like a table of the genes shared among increased scn ko and het in the above plot along with the increased retina (e.g. the 269 and 103 gene sets).

23.3 scn vs retina, p15

table_names <- c("sr_p15_het", "sr_p15_ko")
table_names %in% names(location_sig)

## [1] FALSE FALSE

location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/sr_p15_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

print(scn_retina_p15_upset_result)

## Error:
## ! object 'scn_retina_p15_upset_result' not found

pp(file = "images/test_location_sr_p15_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

23.4 dlgn vs retina, p08

## The appropriate data structure is 'genotype_tables',
## and the tables of interest are:
table_names <- c("dr_p08_het", "dr_p08_ko")
location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/dr_p08_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

pp(file = "images/test_location_dr_p08_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

23.5 dlgn vs retina, p15

## The appropriate data structure is 'genotype_tables',
## and the tables of interest are:
table_names <- c("dr_p15_het", "dr_p15_ko")
location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/dr_p15_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

pp(file = "images/test_location_dr_p15_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

23.6 dlgn vs scn, p08

table_names <- c("ds_p08_het", "ds_p08_ko")
location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/ds_p08_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

pp(file = "images/test_location_ds_p08_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

23.7 dlgn vs scn, p15

table_names <- c("ds_p15_het", "ds_p15_ko")
location_upset_input <- list()
first_table <- table_names[1]
newsig <- location_sig[[first_table]]
for (sig in 2:length(table_names)) {
  name <- table_names[sig]
  newsig[["deseq"]][["ups"]][[name]] <- location_sig[[name]][["deseq"]][["ups"]][[name]]
  newsig[["deseq"]][["downs"]][[name]] <- location_sig[[name]][["deseq"]][["downs"]][[name]]
}
location_upsetr <- upsetr_sig(newsig)

## Error in `xtfrm.data.frame()`:
## ! cannot xtfrm data frames

location_upset_written <- write_upset_groups(location_upsetr, excel = "excel/ds_p15_hetko_upset_groups.xlsx")

## Error:
## ! object 'location_upsetr' not found

location_upsetr[["all_plot"]]

## Error:
## ! object 'location_upsetr' not found

pp(file = "images/test_location_ds_p15_hetko_upset.pdf")
print(location_upsetr[["all_plot"]])

## Error:
## ! object 'location_upsetr' not found

plotted <- dev.off()

24 Shared and unique gene sets across x/wt

In this block I want to find the unique and shared genes between:

1. scn p8 het/wt and retina p8 het/wt: hwp08scninc, hwp08retinc, hwp08scndec, hwp08retdec
2. scn p15 het/wt and retina p15 het/wt: hwp15scninc, hwp15retinc, hwp15scndec, hwp15retdec
3. #1 and #2 together: 8 catgories above
4. scn p8 ko/wt and retina p8 ko/wt
5. scn p15 ko/wt and retina p15 ko/wt
6. #4 and #5 together

The comparisons of het/wt are found in the ‘inclusion_sig’ dataset; because they are providing our cutoffs for nonspecific binding.

24.1 Number 1 above: p08_het vs wt for scn and retina.

table_names <- c("p08_het_scn", "p08_het_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)
inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p08_het_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p08_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

24.2 Number 2 above: p15_het vs wt for scn and retina.

table_names <- c("p15_het_scn", "p15_het_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)

## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?

inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p15_het_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p15_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

24.3 Number 3 above: combination of #1 and #2

table_names <- c("p08_het_scn", "p08_het_retina", "p15_het_scn", "p15_het_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)
inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p08p15_het_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p08p15_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

24.4 Number 4 above p08_ko vs wt for scn and retina.

table_names <- c("p08_ko_scn", "p08_ko_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)
inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p08_ko_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p08_ko_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

24.5 Number 5 above p15_ko vs wt for scn and retina.

table_names <- c("p15_ko_scn", "p15_ko_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)

## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?
## `geom_line()`: Each group consists of only one observation.
## i Do you need to adjust the group aesthetic?

inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p15_ko_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p15_ko_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

24.6 Number 6 above: Combining #4 and #5

table_names <- c("p08_ko_scn", "p08_ko_retina", "p15_ko_scn", "p15_ko_retina")
inclusion_upsetr <- upsetr_sig(inclusion_sig, contrasts = table_names)
inclusion_upset_written <- write_upset_groups(inclusion_upsetr, excel = "excel/rs_p08p15_ko_inclusion_upset_groups.xlsx")
inclusion_upsetr[["all_plot"]]

pp(file = "images/inclusion_sr_p08p15_ko_upset.pdf")
print(inclusion_upsetr[["all_plot"]])
plotted <- dev.off()

25 GSVA

msigdb <- "reference/msigdb_v2024.1.Mm.db"
if (file.exists(msigdb)) {
  v3_h_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "mh",
                           signatures = msigdb, id_source = "fdata",
                           required_id = "mgi_symbol")
  v3_h_gsva

  v3_h_gsva_sig <- get_sig_gsva_categories(
    v3_h_gsva, excel = "excel/gsva_sig_hallmark_categories.xlsx")
  v3_h_gsva_sig

  v3_m1_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "m1",
                            signatures = msigdb, id_source = "fdata",
                            required_id = "mgi_symbol")
  v3_m1_gsva
  v3_m1_gsva_sig <- get_sig_gsva_categories(
    v3_m1_gsva, excel = "excel/gsva_sig_positional_categories.xlsx")
  v3_m1_gsva_sig

  v3_m2_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "m2",
                            signatures = msigdb, id_source = "fdata",
                            required_id = "mgi_symbol")
  v3_m2_gsva
  v3_m2_gsva_sig <- get_sig_gsva_categories(
    v3_m2_gsva, excel = "excel/gsva_sig_curated_categories.xlsx")
  v3_m2_gsva_sig

  v3_m3_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "m3",
                            signatures = msigdb, id_source = "fdata",
                            required_id = "mgi_symbol")
  v3_m3_gsva
  v3_m3_gsva_sig <- get_sig_gsva_categories(
    v3_m3_gsva, excel = "excel/gsva_sig_regulatory_categories.xlsx")
  v3_m3_gsva_sig

  v3_m5_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "m5",
                            signatures = msigdb, id_source = "fdata",
                            required_id = "mgi_symbol")
  v3_m5_gsva
  v3_m5_gsva_sig <- get_sig_gsva_categories(
    v3_m5_gsva, excel = "excel/gsva_sig_ontology_categories.xlsx")
  v3_m5_gsva_sig

  v3_m8_gsva <- simple_gsva(v3_pairwise_input, orgdb = "org.Mm.eg.db", signature_category = "m8",
                            signatures = msigdb, id_source = "fdata",
                            required_id = "mgi_symbol")
  v3_m8_gsva
  v3_m8_gsva_sig <- get_sig_gsva_categories(
    v3_m8_gsva, excel = "excel/gsva_sig_celltype_categories.xlsx")
  v3_m8_gsva_sig
}

26 GSEA images

Up above I created a fairly large set of enrichment/GSEA analyses. Let us pull some of the most interesting results here and look at them.

Here are the specific queries from Rashmi:

• Genotype (het vs ko):
  • P8 het and ko for Ret
  • SCN (P8 het vs KO SCN
  • P8 het vs KO Ret)
  • P15 het and ko for Ret
• Location (somal vs axonal):
  • SR_P08_KO
  • SR_P08_Het
  • SR_P15_KO
  • SR_P15_Het
• Time(p8vs p15):
  • t_het_Ret_ po8-p15
  • t_KO_Ret_C po8-p15
  • t_het_SCN_po8-p15
  • t_KO_SCN_po8-p15

26.1 Genotype

Let us take a moment and see for which contrasts I acquired results:

I need to make a little summary for clusterprofiler too so that I can easily see how many hits there are for each contrast.

summary(genotype_full_gp)

## Error:
## ! object 'genotype_full_gp' not found

for (i in names(genotype_full_gp)) {
  print(i)
  print(genotype_full_gp[[i]][["num_hits"]])
}

## Error:
## ! object 'genotype_full_gp' not found

summary(genotype_full_cp)

## Error:
## ! object 'genotype_full_cp' not found

for (i in names(genotype_full_cp)) {
  print(i)
  print(nrow(genotype_full_cp[[i]][["gse_go"]]))
}

## Error:
## ! object 'genotype_full_cp' not found

26.1.1 p8 het/ko for retina:

This contrast, even before filtering away the high-wt genes, only has 8 genes in the set of up and down genes combined. As a result, my function which performs gProfiler/clusterProfiler skips it, and also skips the p15 het/ko for retina samples.

26.1.2 p8 het/ko for scn:

This has a bunch more genes: 51 up and 128 down. Unfortunately, gProfiler sees no significant over-representation in the up category of genes. The down category has

The up/down sets from clusterProfiler have enrich_go, gse_go, and enrich_objects to look at.

genotype_full_gp$kh_p08_scn_up$num_hits

## Error:
## ! object 'genotype_full_gp' not found

genotype_full_gp$kh_p08_scn_down$num_hits

## Error:
## ! object 'genotype_full_gp' not found

plots <- plot_enrichresult(genotype_full_gp$kh_p08_scn_down[["BP_enrich"]])

## Error:
## ! object 'genotype_full_gp' not found

plots[["dot"]]

## Error:
## ! object 'plots' not found

plots[["tree"]]

## Error:
## ! object 'plots' not found

Perhaps I should just ask the question: for which categories did I get results back?

summary(genotype_full_gp)

## Error:
## ! object 'genotype_full_gp' not found

kh_p08_dlgn_up: No significant gProfiler results. kh_p15_dlgn_up: Significant BP, HP, KEGG, MF, REAC, TF kh_p08_scn_up: No significant gProfiler results. kh_p08_scn_down: Significant BP, MiRNA, MF, TF kh_p15_scn_down: Significant BP, MF

plots <- plot_enrichresult(genotype_full_gp[["kh_p15_dlgn_up"]][["BP_enrich"]])

## Error:
## ! object 'genotype_full_gp' not found

plots[["dot"]]

## Error:
## ! object 'plots' not found

26.2 Location

26.2.1 Scn vs retina ko, p08

plots <- plot_enrichresult(location_gp[["sr_p08_ko"]][["sr_p08_ko_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(location_gp[["sr_p08_ko"]][["sr_p08_ko_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

26.2.2 scn vs retina, het, p08

Enriched groups: BP, KEGG, MF, TF, CC

summary(location_gp[["sr_p08_het"]][["sr_p08_het_up"]])

## Length  Class   Mode 
##      0   NULL   NULL

plots <- plot_enrichresult(location_gp[["sr_p08_het"]][["sr_p08_het_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(location_gp[["sr_p08_het"]][["sr_p08_het_up"]][["CC_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(location_gp[["sr_p08_het"]][["sr_p08_het_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

sr_p08_het_topn_gsea <- plot_topn_gsea(location_cp[[""]])

## Error in `if (nrow(gse) < topn) ...`:
## ! argument is of length zero

26.2.3 Scn vs retina ko, p15

plots <- plot_enrichresult(location_gp[["sr_p15_ko"]][["sr_p15_ko_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(location_gp[["sr_p15_ko"]][["sr_p15_ko_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

26.2.4 scn vs retina, het, p15

plots <- plot_enrichresult(location_gp[["sr_p15_het"]][["sr_p15_het_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(location_gp[["sr_p15_het"]][["sr_p15_het_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

26.3 Time

26.3.1 het retina

Ups: significant results for BP, MF, TF Downs: BP, MF, REAC, TF, WP

plots <- plot_enrichresult(time_gp[["t_het_retina"]][["t_het_retina_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(time_gp[["t_het_retina"]][["t_het_retina_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

26.3.2 ko retina

Up: BP, MiRNA, MF Down: BP, MF, REAC, TF

plots <- plot_enrichresult(time_gp[["t_ko_retina"]][["t_ko_retina_up"]][["BP_enrich"]])
plots[["dot"]]

## NULL

plots <- plot_enrichresult(time_gp[["t_ko_retina"]][["t_ko_retina_down"]][["BP_enrich"]])
plots[["dot"]]

## NULL

26.3.3 het scn

Neither of the SCN gProfiler queries provided any results.

27 Bibliography

pander::pander(sessionInfo())
message(paste0("This is hpgltools commit: ", get_git_commit()))
message(paste0("Saving to ", savefile))
tmp <- sm(saveme(filename = savefile))

tmp <- loadme(filename = savefile)

Hänzelmann, Sonja, Robert Castelo, and Justin Guinney. 2013. “GSVA: Gene Set Variation Analysis for Microarray and RNA-Seq Data.” BMC Bioinformatics 14 (1): 7. https://doi.org/10.1186/1471-2105-14-7.

Hoffman, Gabriel E, and Panos Roussos. 2020. “Dream: Powerful Differential Expression Analysis for Repeated Measures Designs.” Bioinformatics 37 (2): 192–201. https://doi.org/10.1093/bioinformatics/btaa687.

Hoffman, Gabriel E., and Eric E. Schadt. 2016. “variancePartition: Interpreting Drivers of Variation in Complex Gene Expression Studies.” BMC Bioinformatics 17 (1): 483. https://doi.org/10.1186/s12859-016-1323-z.

Leek, Jeffrey T., W. Evan Johnson, Hilary S. Parker, Andrew E. Jaffe, and John D. Storey. 2012. “The SVA Package for Removing Batch Effects and Other Unwanted Variation in High-Throughput Experiments.” Bioinformatics 28 (6): 882–83. https://doi.org/10.1093/bioinformatics/bts034.

Leng, Ning, John A. Dawson, James A. Thomson, Victor Ruotti, Anna I. Rissman, Bart M. G. Smits, Jill D. Haag, Michael N. Gould, Ron M. Stewart, and Christina Kendziorski. 2013. “EBSeq: An Empirical Bayes Hierarchical Model for Inference in RNA-seq Experiments.” Bioinformatics 29 (8): 1035–43. https://doi.org/10.1093/bioinformatics/btt087.

Li, Huiling, Yangao Huo, Xi He, Liping Yao, Hao Zhang, Yiqiang Cui, Huijuan Xiao, et al. 2022. “A Male Germ-Cell-Specific Ribosome Controls Male Fertility.” Nature 612 (7941): 725–31. https://doi.org/10.1038/s41586-022-05508-0.

Love, Michael I., Wolfgang Huber, and Simon Anders. 2014. “Moderated Estimation of Fold Change and Dispersion for RNA-Seq Data with DESeq2.” bioRxiv. https://doi.org/10.1101/002832.

Raudvere, Uku, Liis Kolberg, Ivan Kuzmin, Tambet Arak, Priit Adler, Hedi Peterson, and Jaak Vilo. 2019. “G:Profiler: A Web Server for Functional Enrichment Analysis and Conversions of Gene Lists (2019 Update).” Nucleic Acids Research 47 (W1): W191–98. https://doi.org/10.1093/nar/gkz369.

Ritchie, Matthew E., Belinda Phipson, Di Wu, Yifang Hu, Charity W. Law, Wei Shi, and Gordon K. Smyth. 2015. “Limma Powers Differential Expression Analyses for RNA-sequencing and Microarray Studies.” Nucleic Acids Research 43 (7): e47. https://doi.org/10.1093/nar/gkv007.

Robinson, Mark D., Davis J. McCarthy, and Gordon K. Smyth. 2010. “edgeR: A Bioconductor Package for Differential Expression Analysis of Digital Gene Expression Data.” Bioinformatics 26 (1): 139–40. https://doi.org/10.1093/bioinformatics/btp616.

Smedley, Damian, Syed Haider, Benoit Ballester, Richard Holland, Darin London, Gudmundur Thorisson, and Arek Kasprzyk. 2009. “BioMart – Biological Queries Made Easy.” BMC Genomics 10 (1): 22. https://doi.org/10.1186/1471-2164-10-22.

Tarazona, Sonia, Fernando García, Alberto Ferrer, Joaquín Dopazo, and Ana Conesa. 2011. “NOIseq: A RNA-seq Differential Expression Method Robust for Sequencing Depth Biases.” EMBnet.journal 17 (B): 18–19. https://doi.org/10.14806/ej.17.B.265.

Yu, Guangchuang. n.d. 📖 Introduction Biomedical Knowledge Mining Using GOSemSim and clusterProfiler. Accessed June 21, 2024.