1 Introduction
This document is intended to create the data structures used to evaluate our TMRC2 samples. In some cases, this includes only those samples starting in 2019; in other instances I am including our previous (2015-2016) samples.
In all cases the processing performed was:
- Default trimming was performed.
- Hisat2 was used to map the remaining reads against the Leishmania panamensis genome revision 36.
- The alignments from hisat2 were used to count reads/gene against the revision 36 annotations with htseq.
- These alignments were also passed to the pileup functionality of samtools and the vcf/bcf utilities in order to make a matrix of all observed differences between each sample with respect to the reference.
- The freebayes variant estimation tool was used in addition to #4 to search for variant positions in a more robust fashion.
- The trimmed reads were passed to kraken2 using a viral database in order to look for samples with potential LRV sequence.
- An explicit, grep-based search for spliced leader reads was used against all human-derived samples. The results from this were copy/pasted into the sample sheet.
1.1 Multiple datasets
In a couple of important ways the TMRC2 data is much more complex than the TMRC3:
- It comprises multiple, completely separate queries:
- Sequencing the parasite samples
- Sequencing a set of human macrophage samples which were infected with specific parasite samples.
- The parasite transcriptomic samples comprise multiple different
types of queries:
- Differential expression to look at strain, susceptibility, and clinical outcomes.
- Individual variant searches to look for potentially useful SNPs for classification of parasite samples.
- The human macrophage samples may be used to query both the host and parasite transcriptomes because (at least when not drug treated) there is a tremendous population of parasite reads in them.
2 Annotations
Everything which follows depends on the Existing TriTrypDB annotations revision 46, circa 2019. The following block loads a database of these annotations and turns it into a matrix where the rows are genes and columns are all the annotation types provided by TriTrypDB.
The same database was used to create a matrix of orthologous genes between L.panamensis and all of the other species in the TriTrypDB.
The same database of annotations also provides mappings to the set of annotated GO categories for the L.panamensis genome along with gene lengths.
meta <- download_eupath_metadata(webservice = "tritrypdb", overwrite = FALSE)
panamensis_entry <- get_eupath_entry("MHOM", metadata = meta[["valid"]])
panamensis_db <- make_eupath_orgdb(panamensis_entry)
panamensis_pkg <- panamensis_db[["pkgname"]]
package_name <- panamensis_db[["pkgname"]]
if (is.null(panamensis_pkg)) {
panamensis_pkg <- panamensis_entry[["OrgdbPkg"]]
package_name <- panamensis_pkg
}
tt <- library(panamensis_pkg, character.only = TRUE)
panamensis_env <- get0(panamensis_pkg)
all_fields <- columns(panamensis_env)
all_lp_annot <- sm(load_orgdb_annotations(
panamensis_env,
keytype = "gid",
fields = c("annot_gene_entrez_id", "annot_gene_name",
"annot_strand", "annot_chromosome", "annot_cds_length",
"annot_gene_product")))$genes
## Testing to see just how big the full database is.
## testing <- load_orgdb_annotations(panamensis_pkg, keytype = "gid", fields = "all")
lp_go <- load_orgdb_go(panamensis_pkg)
lp_go <- lp_go[, c("GID", "GO")]
lp_lengths <- all_lp_annot[, c("gid", "annot_cds_length")]
colnames(lp_lengths) <- c("ID", "length")
all_lp_annot[["annot_gene_product"]] <- tolower(all_lp_annot[["annot_gene_product"]])
orthos <- sm(extract_eupath_orthologs(db = panamensis_pkg))
data_structures <- c(data_structures, "lp_lengths", "lp_go", "all_lp_annot")all_installed <- rownames(installed.packages())
candidates <- grepl(pattern = "^org.Lpanamensis.MHOM.*v68.eg.db", x = all_installed)
do_parasite <- FALSE
if (sum(candidates) > 0) {
do_parasite <- TRUE
orgdb_pkg_name <- all_installed[candidates]
tt <- library(orgdb_pkg_name, character.only = TRUE)
panamensis_pkg <- get0(orgdb_pkg_name)
all_fields <- columns(panamensis_pkg)
all_lp_annot <- sm(load_orgdb_annotations(
panamensis_pkg,
keytype = "gid",
fields = c("annot_gene_entrez_id", "annot_gene_name",
"annot_strand", "annot_chromosome", "annot_cds_length",
"annot_gene_product")))$genes
lp_go <- load_orgdb_go(panamensis_pkg)
lp_go <- lp_go[, c("GID", "GO")]
lp_lengths <- all_lp_annot[, c("gid", "annot_cds_length")]
colnames(lp_lengths) <- c("ID", "length")
all_lp_annot[["annot_gene_product"]] <- tolower(all_lp_annot[["annot_gene_product"]])
data_structures <- c(data_structures, "lp_lengths", "lp_go", "all_lp_annot")
}## Loading required package: AnnotationDbi## Loading required package: stats4## Loading required package: BiocGenerics## Loading required package: generics##
## Attaching package: 'generics'## The following object is masked from 'package:dplyr':
##
## explain## The following objects are masked from 'package:base':
##
## as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
## setequal, union##
## Attaching package: 'BiocGenerics'## The following object is masked from 'package:dplyr':
##
## combine## The following objects are masked from 'package:hpgltools':
##
## conditions, conditions<-, normalize## The following objects are masked from 'package:stats':
##
## IQR, mad, sd, var, xtabs## The following objects are masked from 'package:base':
##
## Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
## as.data.frame, basename, cbind, colnames, dirname, do.call,
## duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
## mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
## rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
## unsplit, which.max, which.min## Loading required package: Biobase## Welcome to Bioconductor
##
## Vignettes contain introductory material; view with
## 'browseVignettes()'. To cite Bioconductor, see
## 'citation("Biobase")', and for packages 'citation("pkgname")'.## Loading required package: IRanges## Loading required package: S4Vectors##
## Attaching package: 'S4Vectors'## The following objects are masked from 'package:dplyr':
##
## first, rename## The following object is masked from 'package:utils':
##
## findMatches## The following objects are masked from 'package:base':
##
## I, expand.grid, unname##
## Attaching package: 'IRanges'## The following objects are masked from 'package:dplyr':
##
## collapse, desc, slice## The following object is masked from 'package:glue':
##
## trim##
## Attaching package: 'AnnotationDbi'## The following object is masked from 'package:dplyr':
##
## select## ## The chosen keytype was not available. Using 'GID'.## This is an orgdb, good.## 'select()' returned 1:many mapping between keys and columns
## 'select()' returned 1:many mapping between keys and columns3 Load a genome
The following block loads the full genome sequence for panamensis. We may use this later to attempt to estimate PCR primers to discern strains.
I am not sure how to increase the number of open files in a container, as a result this does not work.
4 Generate Expressionsets and Sample Estimation
The process of sample estimation takes two primary inputs:
- The sample sheet, which contains all the metadata we currently have on hand, including filenames for the outputs of #3 and #4 above.
- The gene annotations.
An expressionSet(or summarizedExperiment) is a data structure used in R to examine RNASeq data. It is comprised of annotations, metadata, and expression data. In the case of our processing pipeline, the location of the expression data is provided by the filenames in the metadata.
4.1 Define colors
The following list contains the colors we have chosen to use when plotting the various ways of discerning the data.
color_choices <- list(
"strain" = list(
## "z1.0" = "#333333", ## Changed this to 'braz' to make it easier to find them.
"z2.0" = "#555555",
"z3.0" = "#777777",
"z2.1" = "#874400",
"z2.2" = "#0000cc",
"z2.3" = "#cc0000",
"z2.4" = "#df7000",
"z3.2" = "#888888",
"z1.0" = "#cc00cc",
"z1.5" = "#cc00cc",
"b2904" = "#cc00cc",
"unknown" = "#cbcbcb"),
## "null" = "#000000"),
"zymo" = list(
"none" = "#000000",
"z22" = "#0000cc",
"z23" = "#cc0000"),
"cf" = list(
"cure" = "#006f00",
"fail" = "#9dffa0",
"unknown" = "#cbcbcb",
"notapplicable" = "#000000"),
"condition" = list(
"inf" = "#199c75",
"inf_sb" = "#d65d00",
"uninf" = "#6e6ea3",
"uninf_sb" = "#d83956"),
"significance" = list(
"lt0" = "#ffe0e0",
"lt1" = "#ffa0a0",
"lt2" = "#f94040",
"lt4" = "#a00000",
"gt0" = "#eeccf9",
"gt1" = "#de8bf9",
"gt2" = "#ad07e3",
"gt4" = "#410257"),
"drug" = list(
"none" = "#989898",
"antimony" = "#088b64"),
"oldnew" = list(
"previous" = "#2233aa",
"current" = "#9c0303"),
"infectedp" = list(
"uninfected" = "#676767",
"infected" = "#ac06e2"),
"treatment_zymo" = list(
"inf_sb_z23" = "#E7298A",
"inf_z23" = "#D95F02",
"uninf_none" = "#66A61E",
"uninf_sb_none" = "#E6AB02",
"inf_z22" = "#1B9E77",
"inf_sb_z22" = "#7570B3"),
"donor" = list(
"d01" = "#8d0000",
"d02" = "#E6AB02",
"d09" = "#7570B3",
"d81" = "#2233aa"),
"susceptibility" = list(
"resistant" = "#8563a7",
"sensitive" = "#8d0000",
"ambiguous" = "#cbcbcb",
"unknown" = "#555555"))
data_structures <- c(data_structures, "color_choices")5 Macrophage data
All of the above focused entire on the parasite samples, now let us pull up the macrophage infected samples. This will comprise two datasets, one of the human and one of the parasite.
5.1 Macrophage host data
The metadata for the macrophage samples contains a couple of columns for mapped human and parasite reads. We will therefore use them separately to create two expressionsets, one for each species.
** Note **: I forgot to commit the addition of plot_metadata factors() in the last run of this. In addition, I need to add an explicit month to load_biomart_annotations() or change the function to search a couple more months before it stops trying to find an archive.
## Using mart: ENSEMBL_MART_ENSEMBL from host: apr2020.archive.ensembl.org.## Successfully connected to the hsapiens_gene_ensembl database.## Finished downloading ensembl gene annotations.## Finished downloading ensembl structure annotations.## Including symbols, there are 67159 vs the 249740 gene annotations.## Not dropping haplotype chromosome annotations, set drop_haplotypes = TRUE if this is bad.## Saving annotations to hsapiens_biomart_annotations.rda.## Finished save().hs_annot <- hs_annot[["annotation"]]
hs_annot[["transcript"]] <- paste0(rownames(hs_annot), ".", hs_annot[["transcript_version"]])
rownames(hs_annot) <- make.names(hs_annot[["ensembl_gene_id"]], unique = TRUE)
rownames(hs_annot) <- paste0("gene:", rownames(hs_annot))
tx_gene_map <- hs_annot[, c("transcript", "ensembl_gene_id")]
sanitize_columns <- c("drug", "macrophagetreatment", "macrophagezymodeme")
macr_annot <- hs_annot
rownames(macr_annot) <- gsub(x = rownames(macr_annot),
pattern = "^gene:",
replacement = "")
hs_macrophage <- create_se(sample_sheet, gene_info = macr_annot,
file_column = "hg38100hisatfile") %>%
set_conditions(fact = "macrophagetreatment") %>%
set_batches(fact = "macrophagezymodeme") %>%
sanitize_metadata(columns = sanitize_columns) %>%
subset_se(nonzero = 12000)## Reading the sample metadata.## Did not find the condition column in the sample sheet.## Filling it in as undefined.## Did not find the batch column in the sample sheet.## Filling it in as undefined.## Checking the state of the condition column.## Checking the state of the batch column.## Checking the condition factor.## The sample definitions comprises: 69 rows(samples) and 86 columns(metadata fields).## Matched 21481 annotations and counts.## Some annotations were lost in merging, setting them to 'undefined'.## The final summarized experiment has 21481 rows and 86 columns.## The numbers of samples by condition are:##
## inf inf_sb uninf uninf_sb
## 30 29 5 5## The number of samples by batch are:##
## none z2.2 z2.3
## 10 30 29## Recasting the data.frame to DataFrame.## sampleid tubelabelorigin samplename numberofvials
## Length:69 Length:69 Length:69 Min. :1
## Class :character Class :character Class :character 1st Qu.:1
## Mode :character Mode :character Mode :character Median :1
## Mean :1
## 3rd Qu.:1
## Max. :1
##
## sourcelab expperson cellssource lvpanamensiszymodeme
## Length:69 Length:69 Length:69 Min. :2.20
## Class :character Class :character Class :character 1st Qu.:2.20
## Mode :character Mode :character Mode :character Median :2.20
## Mean :2.25
## 3rd Qu.:2.30
## Max. :2.30
## NA's :22
## lvpanamensissbvsusceptibility samplecollectiondate experimentalbatch
## Length:69 Min. :20190629 Length:69
## Class :character 1st Qu.:20210813 Class :character
## Mode :character Median :20220827 Mode :character
## Mean :20212685
## 3rd Qu.:20220827
## Max. :20220916
##
## typeofcells donor isolationmethod
## Length:69 Length:69 Length:69
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## cellspurificationmethod selectionmethod rnapreservation
## Length:69 Length:69 Length:69
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## rnaextractiondate rnavolumeul rnaavailableul rnaqctesteddate
## Length:69 Min. :30 Min. : 5.2 Min. : 42382
## Class :character 1st Qu.:30 1st Qu.:22.2 1st Qu.:20200114
## Mode :character Median :30 Median :23.3 Median :20205521
## Mean :30 Mean :23.1 Mean :19485696
## 3rd Qu.:30 3rd Qu.:25.6 3rd Qu.:20211111
## Max. :30 Max. :26.2 Max. :20211221
## NA's :41 NA's :41 NA's :41
## bioanalyzerrnangul rnaqcpassed rin nanodroprnangul
## Length:69 Length:69 Length:69 Length:69
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
##
## x260280 x260230 rnausedtoconstructlibrariesul
## Length:69 Length:69 Min. : 2.20
## Class :character Class :character 1st Qu.: 2.80
## Mode :character Mode :character Median : 3.41
## Mean : 4.61
## 3rd Qu.: 4.83
## Max. :23.30
## NA's :41
## rnausedtoconstructlibrariesng libraryqctesteddate libqcpassed
## Min. : 0.5 Min. :20200115 Length:69
## 1st Qu.: 0.5 1st Qu.:20200115 Class :character
## Median :250.3 Median :20201221 Mode :character
## Mean :275.3 Mean :20205505
## 3rd Qu.:500.0 3rd Qu.:20211219
## Max. :800.0 Max. :20211223
## NA's :41 NA's :42
## index libraryvolumeul libraryvolumesenttonajibslabul
## Min. : 1.0 Min. :28 Min. :15
## 1st Qu.: 7.0 1st Qu.:28 1st Qu.:15
## Median :14.5 Median :28 Median :15
## Mean :13.9 Mean :28 Mean :15
## 3rd Qu.:20.2 3rd Qu.:28 3rd Qu.:15
## Max. :27.0 Max. :28 Max. :15
## NA's :41 NA's :41 NA's :41
## shipmentdate oldnew countersampleatcideimul drug
## Min. :20200217 Length:69 Min. :13 antimony:34
## 1st Qu.:20200217 Class :character 1st Qu.:13 none :35
## Median :20210558 Mode :character Median :13
## Mean :20210188 Mean :13
## 3rd Qu.:20220103 3rd Qu.:13
## Max. :20220103 Max. :13
## NA's :41 NA's :56
## descriptonandremarks observation
## Length:69 Length:69
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## librarybioanalyzerprofileelsayedlabfilenamewelllane libraryconcnm
## Length:69 Min. : 22.1
## Class :character 1st Qu.: 57.8
## Mode :character Median : 98.7
## Mean : 96.2
## 3rd Qu.:112.8
## Max. :218.0
## NA's :55
## samplefor100ul2or4nmsequencing waterfor100ul2or4nmsequencing
## Min. :0.917 Min. :92.0
## 1st Qu.:1.754 1st Qu.:96.9
## Median :1.942 Median :98.1
## Mean :2.995 Mean :97.0
## 3rd Qu.:3.082 3rd Qu.:98.2
## Max. :8.000 Max. :99.1
## NA's :56 NA's :56
## sequencingorderno seqorderdate seqcompletedate totalreads
## Length:69 Min. :20200901 Min. :20200910 Min. :12254196
## Class :character 1st Qu.:20200901 1st Qu.:20200910 1st Qu.:21496945
## Mode :character Median :20200901 Median :20200910 Median :23391483
## Mean :20202287 Mean :20202296 Mean :29119440
## 3rd Qu.:20200901 3rd Qu.:20200910 3rd Qu.:27612540
## Max. :20210601 Max. :20210610 Max. :86048061
## NA's :55 NA's :55
## trimmedreads percentkept hg38100salmonfile hg38100hisatfile
## Min. :10400801 Min. :0.735 Length:69 Length:69
## 1st Qu.:19375511 1st Qu.:0.887 Class :character Class :character
## Median :21401552 Median :0.902 Mode :character Mode :character
## Mean :26367248 Mean :0.903
## 3rd Qu.:25048428 3rd Qu.:0.940
## Max. :79775170 Max. :0.945
##
## hisatsinglemappedhg38 hisatmultimappedhg38 hisatmappingratehg38
## Min. : 501881 Min. : 328180 Min. :0.0418
## 1st Qu.:17620997 1st Qu.: 605198 1st Qu.:0.9114
## Median :18986346 Median : 682134 Median :0.9606
## Mean :23074608 Mean :1339401 Mean :0.9223
## 3rd Qu.:22472518 3rd Qu.: 830271 3rd Qu.:0.9787
## Max. :70100660 Max. :7030835 Max. :0.9885
##
## lpanamensisv36hisatfile hisatlpsinglemapped hisatlpmultimapped
## Length:69 Min. : 199 Min. : 14
## Class :character 1st Qu.: 11504 1st Qu.: 770
## Mode :character Median : 284271 Median : 17386
## Mean : 909574 Mean : 70729
## 3rd Qu.:1006681 3rd Qu.: 81008
## Max. :7086371 Max. :620799
##
## parasitemappingrate parasitehostratio x68 macrophagetreatment
## Min. :0.000011 Min. :0.000204 Length:69 inf :30
## 1st Qu.:0.000487 1st Qu.:0.003393 Class :character inf_sb :29
## Median :0.010078 Median :0.003393 Mode :character uninf : 5
## Mean :0.036938 Mean :0.010336 uninf_sb: 5
## 3rd Qu.:0.048453 3rd Qu.:0.003393
## Max. :0.292395 Max. :0.155351
##
## macrophagezymodeme strainid slr1fwd slr1rc
## none:10 Length:69 Min. : 0.00 Min. : 0
## z22 :30 Class :character 1st Qu.: 0.00 1st Qu.: 0
## z23 :29 Mode :character Median : 0.00 Median : 6
## Mean : 1.16 Mean : 31
## 3rd Qu.: 1.00 3rd Qu.: 27
## Max. :11.00 Max. :641
##
## slr2fwd slr2rc slsum slvsreads
## Min. : 2 Min. : 0.000 Min. : 2 Min. :1.03e-07
## 1st Qu.: 224 1st Qu.: 0.000 1st Qu.: 225 1st Qu.:7.81e-06
## Median : 4249 Median : 0.000 Median : 4300 Median :1.83e-04
## Mean : 17256 Mean : 0.667 Mean : 17289 Mean :6.68e-04
## 3rd Qu.: 22089 3rd Qu.: 0.000 3rd Qu.: 22188 3rd Qu.:8.17e-04
## Max. :168826 Max. :13.000 Max. :169487 Max. :6.57e-03
##
## slvshuman trimomatic_input trimomatic_output trimomatic_percent
## Min. :1.05e-07 Length:69 Length:69 Length:69
## 1st Qu.:1.06e-05 Class :character Class :character Class :character
## Median :1.87e-04 Mode :character Mode :character Mode :character
## Mean :8.21e-04
## 3rd Qu.:8.79e-04
## Max. :1.02e-02
##
## fastqc_pct_gc hisat_genome_single_concordant
## Length:69 Length:69
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## hisat_genome_multi_concordant hisat_genome_single_all hisat_genome_multi_all
## Length:69 Length:69 Length:69
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## hisat_genome_percent hisat_count_table manualzymodeme
## Length:69 Length:69 Length:69
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## zymodemevariantcounts zymodemescore condition batch
## Length:69 Length:69 Length:69 Length:69
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
## ## The samples (and read coverage) removed when filtering 12000 non-zero genes are:## TMRC30162
## 521145
## TMRC30162
## 10208## Samples removed: 10208fixed_genenames <- gsub(x = rownames(assay(hs_macrophage)), pattern = "^gene:",
replacement = "")
hs_macrophage <- set_genenames(hs_macrophage, ids = fixed_genenames)
table(colData(hs_macrophage)$condition)##
## inf inf_sb uninf uninf_sb
## 29 29 5 5## Added to make a simplified PCA plot.
colData(hs_macrophage)[["experiment"]] <- "macrophage"
## The following 3 lines were copy/pasted to datastructures and should be removed soon.
nostrain <- is.na(colData(hs_macrophage)[["strainid"]])
colData(hs_macrophage)[nostrain, "strainid"] <- "none"
colData(hs_macrophage)[["strain_zymo"]] <- paste0("s", colData(hs_macrophage)[["strainid"]],
"_", colData(hs_macrophage)[["macrophagezymodeme"]])
uninfected <- colData(hs_macrophage)[["strain_zymo"]] == "snone_none"
colData(hs_macrophage)[uninfected, "strain_zymo"] <- "uninfected"
colData(hs_macrophage)[["infectedp"]] <- "infected"
colData(hs_macrophage)[uninfected, "infectedp"] <- "uninfected"
data_structures <- c(data_structures, "hs_macrophage")5.2 Double-check sample IDs against the sample sheet
1 sample has been excluded from the analysis but is in the sample sheet. I am reasonably certain I know which, but will double-check here.
sample_sheet_ids = c("TMRC30051","TMRC30057","TMRC30059","TMRC30060","TMRC30061","TMRC30062",
"TMRC30063","TMRC30064","TMRC30065","TMRC30066","TMRC30067","TMRC30069",
"TMRC30117","TMRC30162","TMRC30243","TMRC30244","TMRC30245","TMRC30246",
"TMRC30247","TMRC30248","TMRC30249","TMRC30250","TMRC30251","TMRC30252",
"TMRC30266","TMRC30267","TMRC30268","TMRC30286","TMRC30326","TMRC30316",
"TMRC30317","TMRC30322","TMRC30323","TMRC30328","TMRC30318","TMRC30319",
"TMRC30324","TMRC30325","TMRC30320","TMRC30321","TMRC30327","TMRC30312",
"TMRC30297","TMRC30298","TMRC30299","TMRC30300","TMRC30295","TMRC30296",
"TMRC30303","TMRC30304","TMRC30301","TMRC30302","TMRC30314","TMRC30315",
"TMRC30313")
found <- sample_sheet_ids %in% colnames(assay(hs_macrophage))
colnames(assay(hs_macrophage))[!found]5.3 Subset and create different groupings
all_human <- sanitize_metadata(hs_macrophage, columns = "drug") %>%
set_conditions(fact = "drug") %>%
set_batches(fact = "typeofcells")## Recasting the data.frame to DataFrame.## sampleid tubelabelorigin samplename numberofvials
## Length:68 Length:68 Length:68 Min. :1
## Class :character Class :character Class :character 1st Qu.:1
## Mode :character Mode :character Mode :character Median :1
## Mean :1
## 3rd Qu.:1
## Max. :1
##
## sourcelab expperson cellssource lvpanamensiszymodeme
## Length:68 Length:68 Length:68 Min. :2.20
## Class :character Class :character Class :character 1st Qu.:2.20
## Mode :character Mode :character Mode :character Median :2.25
## Mean :2.25
## 3rd Qu.:2.30
## Max. :2.30
## NA's :22
## lvpanamensissbvsusceptibility samplecollectiondate experimentalbatch
## Length:68 Min. :20190629 Length:68
## Class :character 1st Qu.:20210813 Class :character
## Mode :character Median :20220827 Mode :character
## Mean :20213009
## 3rd Qu.:20220827
## Max. :20220916
##
## typeofcells donor isolationmethod
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## cellspurificationmethod selectionmethod rnapreservation
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## rnaextractiondate rnavolumeul rnaavailableul rnaqctesteddate
## Length:68 Min. :30 Min. : 5.2 Min. : 42382
## Class :character 1st Qu.:30 1st Qu.:22.1 1st Qu.:20200114
## Mode :character Median :30 Median :23.3 Median :20210914
## Mean :30 Mean :23.0 Mean :19459236
## 3rd Qu.:30 3rd Qu.:25.4 3rd Qu.:20211111
## Max. :30 Max. :26.2 Max. :20211221
## NA's :41 NA's :41 NA's :41
## bioanalyzerrnangul rnaqcpassed rin nanodroprnangul
## Length:68 Length:68 Length:68 Length:68
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
##
## x260280 x260230 rnausedtoconstructlibrariesul
## Length:68 Length:68 Min. : 2.20
## Class :character Class :character 1st Qu.: 2.80
## Mode :character Mode :character Median : 3.50
## Mean : 4.67
## 3rd Qu.: 4.95
## Max. :23.30
## NA's :41
## rnausedtoconstructlibrariesng libraryqctesteddate libqcpassed
## Min. : 0.5 Min. :20200115 Length:68
## 1st Qu.: 0.5 1st Qu.:20200115 Class :character
## Median :500.0 Median :20205670 Mode :character
## Mean :285.4 Mean :20205669
## 3rd Qu.:500.0 3rd Qu.:20211221
## Max. :800.0 Max. :20211223
## NA's :41 NA's :42
## index libraryvolumeul libraryvolumesenttonajibslabul
## Min. : 1.0 Min. :28 Min. :15
## 1st Qu.: 7.5 1st Qu.:28 1st Qu.:15
## Median :15.0 Median :28 Median :15
## Mean :14.2 Mean :28 Mean :15
## 3rd Qu.:20.5 3rd Qu.:28 3rd Qu.:15
## Max. :27.0 Max. :28 Max. :15
## NA's :41 NA's :41 NA's :41
## shipmentdate oldnew countersampleatcideimul drug
## Min. :20200217 Length:68 Min. :13 antimony:34
## 1st Qu.:20200217 Class :character 1st Qu.:13 none :34
## Median :20211012 Mode :character Median :13
## Mean :20210192 Mean :13
## 3rd Qu.:20220103 3rd Qu.:13
## Max. :20220103 Max. :13
## NA's :41 NA's :55
## descriptonandremarks observation
## Length:68 Length:68
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## librarybioanalyzerprofileelsayedlabfilenamewelllane libraryconcnm
## Length:68 Min. : 25.0
## Class :character 1st Qu.: 64.9
## Mode :character Median :103.0
## Mean :101.9
## 3rd Qu.:114.0
## Max. :218.0
## NA's :55
## samplefor100ul2or4nmsequencing waterfor100ul2or4nmsequencing
## Min. :0.917 Min. :92.0
## 1st Qu.:1.754 1st Qu.:96.9
## Median :1.942 Median :98.1
## Mean :2.995 Mean :97.0
## 3rd Qu.:3.082 3rd Qu.:98.2
## Max. :8.000 Max. :99.1
## NA's :55 NA's :55
## sequencingorderno seqorderdate seqcompletedate totalreads
## Length:68 Min. :20200901 Min. :20200910 Min. :12254196
## Class :character 1st Qu.:20200901 1st Qu.:20200910 1st Qu.:21574725
## Mode :character Median :20200901 Median :20200910 Median :23459956
## Mean :20201647 Mean :20201656 Mean :29237101
## 3rd Qu.:20200901 3rd Qu.:20200910 3rd Qu.:27632525
## Max. :20210601 Max. :20210610 Max. :86048061
## NA's :55 NA's :55
## trimmedreads percentkept hg38100salmonfile hg38100hisatfile
## Min. :10400801 Min. :0.735 Length:68 Length:68
## 1st Qu.:19451334 1st Qu.:0.889 Class :character Class :character
## Median :21493351 Median :0.902 Mode :character Mode :character
## Mean :26501589 Mean :0.904
## 3rd Qu.:25238786 3rd Qu.:0.940
## Max. :79775170 Max. :0.945
##
## hisatsinglemappedhg38 hisatmultimappedhg38 hisatmappingratehg38
## Min. : 501881 Min. : 328180 Min. :0.0418
## 1st Qu.:17798143 1st Qu.: 601364 1st Qu.:0.9194
## Median :19059690 Median : 684508 Median :0.9608
## Mean :23215121 Mean :1350199 Mean :0.9238
## 3rd Qu.:22491433 3rd Qu.: 830880 3rd Qu.:0.9791
## Max. :70100660 Max. :7030835 Max. :0.9885
##
## lpanamensisv36hisatfile hisatlpsinglemapped hisatlpmultimapped
## Length:68 Min. : 199 Min. : 14
## Class :character 1st Qu.: 10778 1st Qu.: 737
## Mode :character Median : 240454 Median : 16033
## Mean : 893249 Mean : 69200
## 3rd Qu.: 968384 3rd Qu.: 74954
## Max. :7086371 Max. :620799
##
## parasitemappingrate parasitehostratio x68 macrophagetreatment
## Min. :0.000011 Min. :0.000204 Length:68 inf :29
## 1st Qu.:0.000467 1st Qu.:0.003393 Class :character inf_sb :29
## Median :0.009909 Median :0.003393 Mode :character uninf : 5
## Mean :0.035609 Mean :0.008203 uninf_sb: 5
## 3rd Qu.:0.048448 3rd Qu.:0.003393
## Max. :0.292395 Max. :0.110394
##
## macrophagezymodeme strainid slr1fwd slr1rc
## none:10 Length:68 Min. : 0.00 Min. : 0.0
## z22 :29 Class :character 1st Qu.: 0.00 1st Qu.: 0.0
## z23 :29 Mode :character Median : 0.00 Median : 5.5
## Mean : 1.18 Mean : 29.8
## 3rd Qu.: 1.00 3rd Qu.: 27.0
## Max. :11.00 Max. :641.0
##
## slr2fwd slr2rc slsum slvsreads
## Min. : 2 Min. : 0.000 Min. : 2 Min. :1.03e-07
## 1st Qu.: 203 1st Qu.: 0.000 1st Qu.: 204 1st Qu.:7.72e-06
## Median : 3885 Median : 0.000 Median : 3916 Median :1.52e-04
## Mean : 16917 Mean : 0.647 Mean : 16948 Mean :6.43e-04
## 3rd Qu.: 20858 3rd Qu.: 0.000 3rd Qu.: 20900 3rd Qu.:7.98e-04
## Max. :168826 Max. :13.000 Max. :169487 Max. :6.57e-03
##
## slvshuman trimomatic_input trimomatic_output trimomatic_percent
## Min. :1.05e-07 Length:68 Length:68 Length:68
## 1st Qu.:9.93e-06 Class :character Class :character Class :character
## Median :1.56e-04 Mode :character Mode :character Mode :character
## Mean :7.91e-04
## 3rd Qu.:8.74e-04
## Max. :1.02e-02
##
## fastqc_pct_gc hisat_genome_single_concordant
## Length:68 Length:68
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## hisat_genome_multi_concordant hisat_genome_single_all hisat_genome_multi_all
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## hisat_genome_percent hisat_count_table manualzymodeme
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## zymodemevariantcounts zymodemescore condition batch
## Length:68 Length:68 inf :29 none:10
## Class :character Class :character inf_sb :29 z2.2:29
## Mode :character Mode :character uninf : 5 z2.3:29
## uninf_sb: 5
##
##
##
## num_nonzero experiment strain_zymo infectedp
## Min. :13978 Length:68 Length:68 Length:68
## 1st Qu.:14558 Class :character Class :character Class :character
## Median :14828 Mode :character Mode :character Mode :character
## Mean :14871
## 3rd Qu.:15159
## Max. :16421
## ## The numbers of samples by condition are:##
## antimony none
## 34 34## The number of samples by batch are:##
## Macrophages U937
## 54 14data_structures <- c(data_structures, "all_human")
## The following 3 lines were copy/pasted to datastructures and should be removed soon.
no_strain_idx <- colData(all_human)[["strainid"]] == "none"
##colData(all_human)[["strainid"]] <- paste0("s", colData(all_human)[["strainid"]],
## "_", colData(all_human)[["macrophagezymodeme"]])
colData(all_human)[no_strain_idx, "strainid"] <- "none"
table(colData(all_human)[["strainid"]])##
## 10763 10772 10977 11026 11075 11126 12251 12309 12355 12367 2169 7158 none
## 2 8 2 2 2 8 7 8 2 7 8 2 10## The numbers of samples by condition are:##
## Macrophages U937
## 54 14## The number of samples by batch are:##
## antimony none
## 34 34data_structures <- c(data_structures, "all_human_types")
type_zymo_fact <- paste0(colData(all_human_types)[["condition"]], "_",
colData(all_human_types)[["macrophagezymodeme"]])
type_zymo <- set_conditions(all_human_types, fact = type_zymo_fact)## The numbers of samples by condition are:##
## Macrophages_none Macrophages_z22 Macrophages_z23 U937_none
## 8 23 23 2
## U937_z22 U937_z23
## 6 6data_structures <- c(data_structures, "type_zymo")
type_drug_fact <- paste0(colData(all_human_types)[["condition"]], "_",
colData(all_human_types)[["drug"]])
type_drug <- set_conditions(all_human_types, fact = type_drug_fact)## The numbers of samples by condition are:##
## Macrophages_antimony Macrophages_none U937_antimony
## 27 27 7
## U937_none
## 7data_structures <- c(data_structures, "type_drug")
strain_fact <- colData(all_human_types)[["strainid"]]
table(strain_fact)## strain_fact
## 10763 10772 10977 11026 11075 11126 12251 12309 12355 12367 2169 7158 none
## 2 8 2 2 2 8 7 8 2 7 8 2 10new_conditions <- paste0(colData(hs_macrophage)[["macrophagetreatment"]], "_",
colData(hs_macrophage)[["macrophagezymodeme"]])
## Note the sanitize() call is redundant with the addition of sanitize() in the
## datastructures file, but I don't want to wait to rerun that.
hs_macr <- set_conditions(hs_macrophage, fact = new_conditions) %>%
sanitize_metadata(column = "drug") %>%
set_se_colors(color_choices[["treatment_zymo"]]) %>%
subset_se(subset = "typeofcells!='U937'")## The numbers of samples by condition are:##
## inf_sb_z22 inf_sb_z23 inf_z22 inf_z23 uninf_none
## 15 14 14 15 5
## uninf_sb_none
## 5## Recasting the data.frame to DataFrame.## sampleid tubelabelorigin samplename numberofvials
## Length:68 Length:68 Length:68 Min. :1
## Class :character Class :character Class :character 1st Qu.:1
## Mode :character Mode :character Mode :character Median :1
## Mean :1
## 3rd Qu.:1
## Max. :1
##
## sourcelab expperson cellssource lvpanamensiszymodeme
## Length:68 Length:68 Length:68 Min. :2.20
## Class :character Class :character Class :character 1st Qu.:2.20
## Mode :character Mode :character Mode :character Median :2.25
## Mean :2.25
## 3rd Qu.:2.30
## Max. :2.30
## NA's :22
## lvpanamensissbvsusceptibility samplecollectiondate experimentalbatch
## Length:68 Min. :20190629 Length:68
## Class :character 1st Qu.:20210813 Class :character
## Mode :character Median :20220827 Mode :character
## Mean :20213009
## 3rd Qu.:20220827
## Max. :20220916
##
## typeofcells donor isolationmethod
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## cellspurificationmethod selectionmethod rnapreservation
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## rnaextractiondate rnavolumeul rnaavailableul rnaqctesteddate
## Length:68 Min. :30 Min. : 5.2 Min. : 42382
## Class :character 1st Qu.:30 1st Qu.:22.1 1st Qu.:20200114
## Mode :character Median :30 Median :23.3 Median :20210914
## Mean :30 Mean :23.0 Mean :19459236
## 3rd Qu.:30 3rd Qu.:25.4 3rd Qu.:20211111
## Max. :30 Max. :26.2 Max. :20211221
## NA's :41 NA's :41 NA's :41
## bioanalyzerrnangul rnaqcpassed rin nanodroprnangul
## Length:68 Length:68 Length:68 Length:68
## Class :character Class :character Class :character Class :character
## Mode :character Mode :character Mode :character Mode :character
##
##
##
##
## x260280 x260230 rnausedtoconstructlibrariesul
## Length:68 Length:68 Min. : 2.20
## Class :character Class :character 1st Qu.: 2.80
## Mode :character Mode :character Median : 3.50
## Mean : 4.67
## 3rd Qu.: 4.95
## Max. :23.30
## NA's :41
## rnausedtoconstructlibrariesng libraryqctesteddate libqcpassed
## Min. : 0.5 Min. :20200115 Length:68
## 1st Qu.: 0.5 1st Qu.:20200115 Class :character
## Median :500.0 Median :20205670 Mode :character
## Mean :285.4 Mean :20205669
## 3rd Qu.:500.0 3rd Qu.:20211221
## Max. :800.0 Max. :20211223
## NA's :41 NA's :42
## index libraryvolumeul libraryvolumesenttonajibslabul
## Min. : 1.0 Min. :28 Min. :15
## 1st Qu.: 7.5 1st Qu.:28 1st Qu.:15
## Median :15.0 Median :28 Median :15
## Mean :14.2 Mean :28 Mean :15
## 3rd Qu.:20.5 3rd Qu.:28 3rd Qu.:15
## Max. :27.0 Max. :28 Max. :15
## NA's :41 NA's :41 NA's :41
## shipmentdate oldnew countersampleatcideimul drug
## Min. :20200217 Length:68 Min. :13 antimony:34
## 1st Qu.:20200217 Class :character 1st Qu.:13 none :34
## Median :20211012 Mode :character Median :13
## Mean :20210192 Mean :13
## 3rd Qu.:20220103 3rd Qu.:13
## Max. :20220103 Max. :13
## NA's :41 NA's :55
## descriptonandremarks observation
## Length:68 Length:68
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## librarybioanalyzerprofileelsayedlabfilenamewelllane libraryconcnm
## Length:68 Min. : 25.0
## Class :character 1st Qu.: 64.9
## Mode :character Median :103.0
## Mean :101.9
## 3rd Qu.:114.0
## Max. :218.0
## NA's :55
## samplefor100ul2or4nmsequencing waterfor100ul2or4nmsequencing
## Min. :0.917 Min. :92.0
## 1st Qu.:1.754 1st Qu.:96.9
## Median :1.942 Median :98.1
## Mean :2.995 Mean :97.0
## 3rd Qu.:3.082 3rd Qu.:98.2
## Max. :8.000 Max. :99.1
## NA's :55 NA's :55
## sequencingorderno seqorderdate seqcompletedate totalreads
## Length:68 Min. :20200901 Min. :20200910 Min. :12254196
## Class :character 1st Qu.:20200901 1st Qu.:20200910 1st Qu.:21574725
## Mode :character Median :20200901 Median :20200910 Median :23459956
## Mean :20201647 Mean :20201656 Mean :29237101
## 3rd Qu.:20200901 3rd Qu.:20200910 3rd Qu.:27632525
## Max. :20210601 Max. :20210610 Max. :86048061
## NA's :55 NA's :55
## trimmedreads percentkept hg38100salmonfile hg38100hisatfile
## Min. :10400801 Min. :0.735 Length:68 Length:68
## 1st Qu.:19451334 1st Qu.:0.889 Class :character Class :character
## Median :21493351 Median :0.902 Mode :character Mode :character
## Mean :26501589 Mean :0.904
## 3rd Qu.:25238786 3rd Qu.:0.940
## Max. :79775170 Max. :0.945
##
## hisatsinglemappedhg38 hisatmultimappedhg38 hisatmappingratehg38
## Min. : 501881 Min. : 328180 Min. :0.0418
## 1st Qu.:17798143 1st Qu.: 601364 1st Qu.:0.9194
## Median :19059690 Median : 684508 Median :0.9608
## Mean :23215121 Mean :1350199 Mean :0.9238
## 3rd Qu.:22491433 3rd Qu.: 830880 3rd Qu.:0.9791
## Max. :70100660 Max. :7030835 Max. :0.9885
##
## lpanamensisv36hisatfile hisatlpsinglemapped hisatlpmultimapped
## Length:68 Min. : 199 Min. : 14
## Class :character 1st Qu.: 10778 1st Qu.: 737
## Mode :character Median : 240454 Median : 16033
## Mean : 893249 Mean : 69200
## 3rd Qu.: 968384 3rd Qu.: 74954
## Max. :7086371 Max. :620799
##
## parasitemappingrate parasitehostratio x68 macrophagetreatment
## Min. :0.000011 Min. :0.000204 Length:68 inf :29
## 1st Qu.:0.000467 1st Qu.:0.003393 Class :character inf_sb :29
## Median :0.009909 Median :0.003393 Mode :character uninf : 5
## Mean :0.035609 Mean :0.008203 uninf_sb: 5
## 3rd Qu.:0.048448 3rd Qu.:0.003393
## Max. :0.292395 Max. :0.110394
##
## macrophagezymodeme strainid slr1fwd slr1rc
## none:10 Length:68 Min. : 0.00 Min. : 0.0
## z22 :29 Class :character 1st Qu.: 0.00 1st Qu.: 0.0
## z23 :29 Mode :character Median : 0.00 Median : 5.5
## Mean : 1.18 Mean : 29.8
## 3rd Qu.: 1.00 3rd Qu.: 27.0
## Max. :11.00 Max. :641.0
##
## slr2fwd slr2rc slsum slvsreads
## Min. : 2 Min. : 0.000 Min. : 2 Min. :1.03e-07
## 1st Qu.: 203 1st Qu.: 0.000 1st Qu.: 204 1st Qu.:7.72e-06
## Median : 3885 Median : 0.000 Median : 3916 Median :1.52e-04
## Mean : 16917 Mean : 0.647 Mean : 16948 Mean :6.43e-04
## 3rd Qu.: 20858 3rd Qu.: 0.000 3rd Qu.: 20900 3rd Qu.:7.98e-04
## Max. :168826 Max. :13.000 Max. :169487 Max. :6.57e-03
##
## slvshuman trimomatic_input trimomatic_output trimomatic_percent
## Min. :1.05e-07 Length:68 Length:68 Length:68
## 1st Qu.:9.93e-06 Class :character Class :character Class :character
## Median :1.56e-04 Mode :character Mode :character Mode :character
## Mean :7.91e-04
## 3rd Qu.:8.74e-04
## Max. :1.02e-02
##
## fastqc_pct_gc hisat_genome_single_concordant
## Length:68 Length:68
## Class :character Class :character
## Mode :character Mode :character
##
##
##
##
## hisat_genome_multi_concordant hisat_genome_single_all hisat_genome_multi_all
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## hisat_genome_percent hisat_count_table manualzymodeme
## Length:68 Length:68 Length:68
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
##
##
##
##
## zymodemevariantcounts zymodemescore condition batch
## Length:68 Length:68 Length:68 none:10
## Class :character Class :character Class :character z2.2:29
## Mode :character Mode :character Mode :character z2.3:29
##
##
##
##
## num_nonzero experiment strain_zymo infectedp
## Min. :13978 Length:68 Length:68 Length:68
## 1st Qu.:14558 Class :character Class :character Class :character
## Median :14828 Mode :character Mode :character Mode :character
## Mean :14871
## 3rd Qu.:15159
## Max. :16421
## data_structures <- c(data_structures, "hs_macr")
ggstats_parasite <- plot_metadata_factors(hs_macr, column = "parasitemappingrate",
type = "ggstats", scale = "log2")
pp(file = "images/ggstats_parasiterate_all_macrophage_drug_treatment.png")
ggstats_parasite
dev.off()## png
## 2
## The numbers of samples by condition are:##
## antimony none
## 27 27hs_macr_strain_se <- set_conditions(hs_macr, fact = "macrophagezymodeme") %>%
subset_se(subset = "macrophagezymodeme != 'none'")## The numbers of samples by condition are:##
## none z22 z23
## 8 23 23##
## 10763 10772 10977 11026 11075 11126 12251 12309 12355 12367 2169 7158 none
## 2 6 2 2 2 6 5 6 2 5 6 2 8Let us see if the sankey plot of these samples looks useful…
ggstats_slreads <- plot_metadata_factors(hs_macrophage, column = "hisatlpsinglemapped",
type = "ggstats", scale = "log2")
pp(file = "images/ggstats_slreads_all_macrophage.png")
ggstats_slreads
dev.off()## png
## 2
ggstats_violin <- plot_metadata_factors(hs_macrophage, column = "hisatlpsinglemapped",
scale = "log2")
ggstats_violin
macr_sankey <- plot_meta_sankey(hs_macrophage, color_choices = color_choices,
factors = c("oldnew", "drug", "infectedp", "macrophagezymodeme"))## Warning: attributes are not identical across measure variables; they will be
## dropped## Warning: The `size` argument of `element_rect()` is deprecated as of ggplot2 3.4.0.
## i Please use the `linewidth` argument instead.
## i The deprecated feature was likely used in the ggsankey package.
## Please report the issue at <https://github.com/davidsjoberg/ggsankey/issues>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.## A sankey plot describing the metadata of 68 samples,
## including 26 out of 0 nodes and traversing metadata factors:
## oldnew, drug, infectedp, macrophagezymodeme.
Finally, split off the U937 samples.
5.4 Macrophage parasite data
In the previous block, we used a new invocation of ensembl-derived annotation data, this time we can just use our existing parasite gene annotations.
if (isTRUE(do_parasite)) {
lp_macrophage <- create_se(
sample_sheet, file_column = "lpanamensisv36hisatfile", gene_info = all_lp_annot,
savefile = glue("rda/lp_macrophage-v{ver}.rda"),
annotation = "org.Lpanamensis.MHOMCOL81L13.v46.eg.db") %>%
set_conditions(fact = "macrophagezymodeme") %>%
set_batches(fact = "macrophagetreatment")
unfilt_written <- write_se(
lp_macrophage,
excel = glue("analyses/macrophage_de/{ver}/read_counts/lp_macrophage_reads_unfiltered-v{ver}.xlsx"))
lp_macrophage_filt <- subset_se(lp_macrophage, nonzero = 2500) %>%
semantic_filter(semantic = c("amastin", "gp63", "leishmanolysin"),
semantic_column = "annot_gene_product")
data_structures <- c(data_structures, "lp_macrophage", "lp_macrophage_filt")
filt_written <- write_se(lp_macrophage_filt,
excel = glue("analyses/macrophage_de/{ver}/read_counts/lp_macrophage_reads_filtered-v{ver}.xlsx"))
lp_macrophage <- lp_macrophage_filt
lp_macrophage_nosb <- subset_se(lp_macrophage, subset="batch!='inf_sb'")
lp_nosb_write <- write_se(
lp_macrophage_nosb,
excel = glue("analyses/macrophage_de/{ver}/read_counts/lp_macrophage_nosb_reads-v{ver}.xlsx"))
data_structures <- c(data_structures, "lp_macrophage_nosb")
spec <- make_rnaseq_spec()
test <- gather_preprocessing_metadata(sample_sheet, specification = spec)
}## Reading the sample metadata.## Did not find the condition column in the sample sheet.## Filling it in as undefined.## Did not find the batch column in the sample sheet.## Filling it in as undefined.## Checking the state of the condition column.## Checking the state of the batch column.## Checking the condition factor.## The sample definitions comprises: 69 rows(samples) and 86 columns(metadata fields).## Warning in create_se(sample_sheet, file_column = "lpanamensisv36hisatfile", :
## Some samples were removed when cross referencing the samples against the count
## data.## Matched 8778 annotations and counts.## The final summarized experiment has 8778 rows and 86 columns.## The numbers of samples by condition are:##
## none z2.2 z2.3
## 8 29 29## The number of samples by batch are:##
## inf inf_sb uninf uninf_sb
## 29 29 4 4## Writing the first sheet, containing a legend and some summary data.## Warning in .local(x, row.names, optional, ...): arguments in '...' ignored## The following samples have less than 5705.7 genes.## [1] "TMRC30066" "TMRC30117" "TMRC30244" "TMRC30246" "TMRC30249" "TMRC30266"
## [7] "TMRC30268" "TMRC30292" "TMRC30300" "TMRC30302" "TMRC30304" "TMRC30309"
## [13] "TMRC30312" "TMRC30313" "TMRC30319" "TMRC30323" "TMRC30325" "TMRC30326"
## [19] "TMRC30327" "TMRC30330" "TMRC30331" "TMRC30332"## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## i Please use `linewidth` instead.
## i The deprecated feature was likely used in the hpgltools package.
## Please report the issue to the authors.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.## Warning in fortify(data, ...): Arguments in `...` must be used.
## x Problematic argument:
## * colour = colors
## i Did you misspell an argument name?## 175550 entries are 0. We are on a log scale, adding 1 to the data.
## Naively calculating coefficient of variation/dispersion with respect to condition.
## Finished calculating dispersion estimates.
## `geom_smooth()` using formula = 'y ~ x'This dataset does not support lmer with condition+batch
## Removing 0 low-count genes (8778 remaining).
## transform_counts: Found 175550 values equal to 0, adding 1 to the matrix.
## `geom_smooth()` using formula = 'y ~ x'The factor none has 8 rows.
## The factor z2.2 has 29 rows.
## The factor z2.3 has 29 rows.
## The samples (and read coverage) removed when filtering 2500 non-zero genes are:## TMRC30066 TMRC30117 TMRC30244 TMRC30246 TMRC30266 TMRC30268 TMRC30304 TMRC30309
## 3080 1147 1662 2834 822 3444 289 188
## TMRC30312 TMRC30313 TMRC30319 TMRC30323 TMRC30325 TMRC30326 TMRC30327 TMRC30330
## 76 96 374 84 356 375 129 181
## TMRC30066 TMRC30117 TMRC30244 TMRC30246 TMRC30266 TMRC30268 TMRC30304 TMRC30309
## 1890 888 1135 1796 649 1915 207 166
## TMRC30312 TMRC30313 TMRC30319 TMRC30323 TMRC30325 TMRC30326 TMRC30327 TMRC30330
## 76 84 270 74 279 303 123 135## Samples removed: 1890, 888, 1135, 1796, 649, 1915, 207, 166, 76, 84, 270, 74, 279, 303, 123, 135
## semantic_expt_filter(): Removed 68 genes.
## Writing the first sheet, containing a legend and some summary data.## Warning in .local(x, row.names, optional, ...): arguments in '...' ignored## The following samples have less than 5661.5 genes.## [1] "TMRC30249" "TMRC30292" "TMRC30300" "TMRC30302" "TMRC30331" "TMRC30332"## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.## Warning in fortify(data, ...): Arguments in `...` must be used.
## x Problematic argument:
## * colour = colors
## i Did you misspell an argument name?## 44583 entries are 0. We are on a log scale, adding 1 to the data.
## Naively calculating coefficient of variation/dispersion with respect to condition.
## Finished calculating dispersion estimates.
## `geom_smooth()` using formula = 'y ~ x'Removing 0 low-count genes (8710 remaining).
## transform_counts: Found 44583 values equal to 0, adding 1 to the matrix.
## `geom_smooth()` using formula = 'y ~ x'The factor z2.2 has 21 rows.
## The factor z2.3 has 29 rows.
## Writing the first sheet, containing a legend and some summary data.## Warning in .local(x, row.names, optional, ...): arguments in '...' ignored## Scale for colour is already present.
## Adding another scale for colour, which will replace the existing scale.
## Scale for fill is already present.
## Adding another scale for fill, which will replace the existing scale.## Warning in fortify(data, ...): Arguments in `...` must be used.
## x Problematic argument:
## * colour = colors
## i Did you misspell an argument name?## 6396 entries are 0. We are on a log scale, adding 1 to the data.## Warning: `aes_string()` was deprecated in ggplot2 3.0.0.
## i Please use tidy evaluation idioms with `aes()`.
## i See also `vignette("ggplot2-in-packages")` for more information.
## i The deprecated feature was likely used in the directlabels package.
## Please report the issue at <https://github.com/tdhock/directlabels/issues>.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.## Naively calculating coefficient of variation/dispersion with respect to condition.
## Finished calculating dispersion estimates.
## `geom_smooth()` using formula = 'y ~ x'The dataset has a minimal or missing set of conditions/batches.
## Removing 119 low-count genes (8591 remaining).
## transform_counts: Found 3163 values equal to 0, adding 1 to the matrix.
## `geom_smooth()` using formula = 'y ~ x'The factor z2.2 has 14 rows.
## The factor z2.3 has 15 rows.
## Did not find the condition column in the sample sheet.
## Filling it in as undefined.
## Did not find the batch column in the sample sheet.
## Filling it in as undefined.
## Checking the state of the condition column.
## Checking the state of the batch column.
## Checking the condition factor.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: trimomatic_input already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: trimomatic_output already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: trimomatic_percent already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: fastqc_pct_gc already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: hisat_genome_single_concordant already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: hisat_genome_multi_concordant already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: hisat_genome_single_all already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: hisat_genome_multi_all already exists, replacing it.## Warning in gather_preprocessing_metadata(sample_sheet, specification = spec):
## Column: hisat_count_table already exists, replacing it.## preprocessing/TMRC30051/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30057/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30059/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30060/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30061/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30062/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30063/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30064/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30065/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30066/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30067/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30069/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30117/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30162/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30243/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30244/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30245/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30246/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30247/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30248/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30249/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30250/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30251/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30252/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30266/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30267/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30268/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30286/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30291/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30292/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30293/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30294/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30295/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30296/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30297/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30298/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30299/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30300/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30301/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30302/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30303/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30304/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30305/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30306/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30307/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30308/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30309/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30310/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30311/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30312/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30313/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30314/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30315/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30316/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30317/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30318/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30319/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30320/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30321/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30322/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30323/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30324/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30325/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30326/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30327/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30328/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30330/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30331/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz
## preprocessing/TMRC30332/outputs/*hisat*_*/*_*genome*_gene_ID_fcounts.csv.xz## Writing new metadata to: sample_sheets/macrophage_samples_modified.xlsx6 Save all data structures into one rda
found_idx <- data_structures %in% ls()
if (sum(!found_idx) > 0) {
not_found <- data_structures[!found_idx]
warning("Some datastructures were not generated: ", toString(not_found), ".")
data_structures <- data_structures[found_idx]
}
save(list = data_structures, file = glue("rda/tmrc2_data_structures-v{ver}.rda"))## Warning: Your system is mis-configured: '/etc/localtime' is not a symlink## Warning: It is strongly recommended to set envionment variable TZ to
## 'America/New_York' (or equivalent)R version 4.5.0 (2025-04-11)
Platform: x86_64-pc-linux-gnu
locale: C
attached base packages: stats4, stats, graphics, grDevices, utils, datasets, methods and base
other attached packages: ruv(v.0.9.7.1), BiocParallel(v.1.42.1), variancePartition(v.1.38.1), org.Lpanamensis.MHOMCOL81L13.v68.eg.db(v.2024.09), AnnotationDbi(v.1.70.0), IRanges(v.2.42.0), S4Vectors(v.0.46.0), Biobase(v.2.68.0), BiocGenerics(v.0.54.0), generics(v.0.1.4), dplyr(v.1.1.4), Heatplus(v.3.16.0), ggplot2(v.4.0.0), glue(v.1.8.0) and hpgltools(v.1.2)
loaded via a namespace (and not attached): fs(v.1.6.6), matrixStats(v.1.5.0), bitops(v.1.0-9), doParallel(v.1.0.17), httr(v.1.4.7), RColorBrewer(v.1.1-3), insight(v.1.4.2), numDeriv(v.2016.8-1.1), tools(v.4.5.0), backports(v.1.5.0), R6(v.2.6.1), statsExpressions(v.1.7.1), lazyeval(v.0.2.2), mgcv(v.1.9-3), withr(v.3.0.2), gridExtra(v.2.3), prettyunits(v.1.2.0), cli(v.3.6.5), performance(v.0.15.1), labeling(v.0.4.3), sass(v.0.4.10), prismatic(v.1.1.2), BWStest(v.0.2.3), mvtnorm(v.1.3-3), S7(v.0.2.0), readr(v.2.1.5), genefilter(v.1.90.0), pbapply(v.1.7-4), Rsamtools(v.2.24.0), yulab.utils(v.0.2.1), DOSE(v.4.2.0), R.utils(v.2.13.0), dichromat(v.2.0-0.1), limma(v.3.64.3), RSQLite(v.2.4.3), BiocIO(v.1.18.0), vroom(v.1.6.5), gtools(v.3.9.5), zip(v.2.3.3), GO.db(v.3.21.0), Matrix(v.1.7-3), abind(v.1.4-8), R.methodsS3(v.1.8.2), lifecycle(v.1.0.4), yaml(v.2.3.10), edgeR(v.4.6.3), SummarizedExperiment(v.1.38.1), gplots(v.3.2.0), qvalue(v.2.40.0), SparseArray(v.1.8.1), BiocFileCache(v.2.16.1), Rtsne(v.0.17), paletteer(v.1.6.0), grid(v.4.5.0), blob(v.1.2.4), promises(v.1.3.3), crayon(v.1.5.3), lattice(v.0.22-7), cowplot(v.1.2.0), GenomicFeatures(v.1.60.0), annotate(v.1.86.1), KEGGREST(v.1.48.1), zeallot(v.0.2.0), pillar(v.1.11.0), knitr(v.1.50), varhandle(v.2.0.6), fgsea(v.1.34.2), GenomicRanges(v.1.60.0), rjson(v.0.2.23), boot(v.1.3-31), corpcor(v.1.6.10), kSamples(v.1.2-12), codetools(v.0.2-20), fastmatch(v.1.1-6), data.table(v.1.17.8), vctrs(v.0.6.5), png(v.0.1-8), Rdpack(v.2.6.4), gtable(v.0.3.6), rematch2(v.2.1.2), datawizard(v.1.2.0), cachem(v.1.1.0), xfun(v.0.53), openxlsx(v.4.2.8), rbibutils(v.2.3), S4Arrays(v.1.8.1), mime(v.0.13), correlation(v.0.8.8), reformulas(v.0.4.1), coda(v.0.19-4.1), survival(v.3.8-3), iterators(v.1.0.14), statmod(v.1.5.0), gmp(v.0.7-5), directlabels(v.2025.6.24), nlme(v.3.1-168), pbkrtest(v.0.5.5), bit64(v.4.6.0-1), EnvStats(v.3.1.0), progress(v.1.2.3), filelock(v.1.0.3), GenomeInfoDb(v.1.44.2), bslib(v.0.9.0), KernSmooth(v.2.23-26), DBI(v.1.2.3), tidyselect(v.1.2.1), bit(v.4.6.0), compiler(v.4.5.0), curl(v.7.0.0), httr2(v.1.2.1), graph(v.1.86.0), xml2(v.1.4.0), DelayedArray(v.0.34.1), plotly(v.4.11.0), bayestestR(v.0.17.0), rtracklayer(v.1.68.0), scales(v.1.4.0), caTools(v.1.18.3), remaCor(v.0.0.20), quadprog(v.1.5-8), multcompView(v.0.1-10), rappdirs(v.0.3.3), stringr(v.1.5.1), digest(v.0.6.37), ggsankey(v.0.0.99999), minqa(v.1.2.8), rmarkdown(v.2.29), aod(v.1.3.3), XVector(v.0.48.0), RhpcBLASctl(v.0.23-42), htmltools(v.0.5.8.1), pkgconfig(v.2.0.3), lme4(v.1.1-37), MatrixGenerics(v.1.20.0), dbplyr(v.2.5.0), fastmap(v.1.2.0), rlang(v.1.1.6), htmlwidgets(v.1.6.4), UCSC.utils(v.1.4.0), shiny(v.1.11.1), SuppDists(v.1.1-9.9), farver(v.2.1.2), jquerylib(v.0.1.4), jsonlite(v.2.0.0), GOSemSim(v.2.34.0), R.oo(v.1.27.1), RCurl(v.1.98-1.17), magrittr(v.2.0.3), GenomeInfoDbData(v.1.2.14), patchwork(v.1.3.2), parameters(v.0.28.2), Rcpp(v.1.1.0), stringi(v.1.8.7), MASS(v.7.3-65), plyr(v.1.8.9), parallel(v.4.5.0), ggrepel(v.0.9.6), Biostrings(v.2.76.0), PMCMRplus(v.1.9.12), splines(v.4.5.0), pander(v.0.6.6), hms(v.1.1.3), locfit(v.1.5-9.12), fastcluster(v.1.3.0), ggsignif(v.0.6.4), effectsize(v.1.0.1), reshape2(v.1.4.4), biomaRt(v.2.64.0), rstantools(v.2.5.0), XML(v.3.99-0.19), evaluate(v.1.0.4), RcppParallel(v.5.1.11-1), tzdb(v.0.5.0), nloptr(v.2.2.1), foreach(v.1.5.2), httpuv(v.1.6.16), MatrixModels(v.0.5-4), BayesFactor(v.0.9.12-4.7), tidyr(v.1.3.1), purrr(v.1.1.0), broom(v.1.0.10), xtable(v.1.8-4), restfulr(v.0.0.16), Rmpfr(v.1.1-1), fANCOVA(v.0.6-1), later(v.1.4.3), viridisLite(v.0.4.2), tibble(v.3.3.0), lmerTest(v.3.1-3), ggstatsplot(v.0.13.2), memoise(v.2.0.1), GenomicAlignments(v.1.44.0), sva(v.3.56.0) and GSEABase(v.1.70.0)
## If you wish to reproduce this exact build of hpgltools, invoke the following:## > git clone http://github.com/abelew/hpgltools.git## > git reset 499a1491ca81e6bda52768411a1ba6e2912d2e74## This is hpgltools commit: Sat Sep 27 13:11:43 2025 -0400: 499a1491ca81e6bda52768411a1ba6e2912d2e74## Saving to 01datasets.rda.xzLS0tCnRpdGxlOiAiTWFjcm9waGFnZSByZXNwb25zZSB0byBkaWZmZXJlbnQgc3RyYWlucyBgciBTeXMuZ2V0ZW52KCdWRVJTSU9OJylgOiBEYXRhIFNldCBDcmVhdGlvbiIKYXV0aG9yOiAiYXRiIGFiZWxld0BnbWFpbC5jb20iCmRhdGU6ICJgciBTeXMuRGF0ZSgpYCIKb3V0cHV0OgogIGh0bWxfZG9jdW1lbnQ6CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICBjb2RlX2ZvbGRpbmc6IHNob3cKICAgIGZpZ19jYXB0aW9uOiB0cnVlCiAgICBmaWdfaGVpZ2h0OiA3CiAgICBmaWdfd2lkdGg6IDcKICAgIGhpZ2hsaWdodDogemVuYnVybgogICAga2VlcF9tZDogZmFsc2UKICAgIG1vZGU6IHNlbGZjb250YWluZWQKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgc2VsZl9jb250YWluZWQ6IHRydWUKICAgIHRoZW1lOiByZWFkYWJsZQogICAgdG9jOiB0cnVlCiAgICB0b2NfZmxvYXQ6CiAgICAgIGNvbGxhcHNlZDogZmFsc2UKICAgICAgc21vb3RoX3Njcm9sbDogZmFsc2UKLS0tCgo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPgpib2R5IC5tYWluLWNvbnRhaW5lciB7CiAgbWF4LXdpZHRoOiAxNjAwcHg7Cn0KYm9keSwgdGQgewogIGZvbnQtc2l6ZTogMTZweDsKfQpjb2RlLnJ7CiAgZm9udC1zaXplOiAxNnB4Owp9CnByZSB7CiAgZm9udC1zaXplOiAxNnB4Cn0KPC9zdHlsZT4KCmBgYHtyIG9wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmxpYnJhcnkoaHBnbHRvb2xzKQoKbGlicmFyeShnbHVlKQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkoSGVhdHBsdXMpCmxpYnJhcnkoZHBseXIpCgprbml0cjo6b3B0c19rbml0JHNldCgKICBwcm9ncmVzcyA9IFRSVUUsIHZlcmJvc2UgPSBUUlVFLCB3aWR0aCA9IDkwLCBlY2hvID0gVFJVRSkKa25pdHI6Om9wdHNfY2h1bmskc2V0KAogIGVycm9yID0gVFJVRSwgZmlnLndpZHRoID0gOCwgZmlnLmhlaWdodCA9IDgsIGZpZy5yZXRpbmEgPSAyLAogIG91dC53aWR0aCA9ICIxMDAlIiwgZGV2ID0gInBuZyIsCiAgZGV2LmFyZ3MgPSBsaXN0KCJwbmciID0gbGlzdCh0eXBlID0gImNhaXJvLXBuZyIpKSkKb2xkX29wdGlvbnMgPC0gb3B0aW9ucygKICBkaWdpdHMgPSA0LCBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UsIGtuaXRyLmR1cGxpY2F0ZS5sYWJlbCA9ICJhbGxvdyIpCmdncGxvdDI6OnRoZW1lX3NldChnZ3Bsb3QyOjp0aGVtZV9idyhiYXNlX3NpemUgPSAxMikpCnZlciA8LSBTeXMuZ2V0ZW52KCJWRVJTSU9OIikKcnVuZGF0ZSA8LSBmb3JtYXQoU3lzLkRhdGUoKSwgZm9ybWF0ID0gIiVZJW0lZCIpCgpybWRfZmlsZSA8LSAiMDFkYXRhc2V0cy5SbWQiCnNhdmVmaWxlIDwtIGdzdWIocGF0dGVybiA9ICJcXC5SbWQiLCByZXBsYWNlID0gIlxcLnJkYVxcLnh6IiwgeCA9IHJtZF9maWxlKQpkYXRhX3N0cnVjdHVyZXMgPC0gYygpCmBgYAoKIyBJbnRyb2R1Y3Rpb24KClRoaXMgZG9jdW1lbnQgaXMgaW50ZW5kZWQgdG8gY3JlYXRlIHRoZSBkYXRhIHN0cnVjdHVyZXMgdXNlZCB0bwpldmFsdWF0ZSBvdXIgVE1SQzIgc2FtcGxlcy4gIEluIHNvbWUgY2FzZXMsIHRoaXMgaW5jbHVkZXMgb25seSB0aG9zZQpzYW1wbGVzIHN0YXJ0aW5nIGluIDIwMTk7IGluIG90aGVyIGluc3RhbmNlcyBJIGFtIGluY2x1ZGluZyBvdXIKcHJldmlvdXMgKDIwMTUtMjAxNikgc2FtcGxlcy4KCkluIGFsbCBjYXNlcyB0aGUgcHJvY2Vzc2luZyBwZXJmb3JtZWQgd2FzOgoKMS4gIERlZmF1bHQgdHJpbW1pbmcgd2FzIHBlcmZvcm1lZC4KMi4gIEhpc2F0MiB3YXMgdXNlZCB0byBtYXAgdGhlIHJlbWFpbmluZyByZWFkcyBhZ2FpbnN0IHRoZSBMZWlzaG1hbmlhCiAgICBwYW5hbWVuc2lzIGdlbm9tZSByZXZpc2lvbiAzNi4KMy4gIFRoZSBhbGlnbm1lbnRzIGZyb20gaGlzYXQyIHdlcmUgdXNlZCB0byBjb3VudCByZWFkcy9nZW5lIGFnYWluc3QgdGhlCiAgICByZXZpc2lvbiAzNiBhbm5vdGF0aW9ucyB3aXRoIGh0c2VxLgo0LiAgVGhlc2UgYWxpZ25tZW50cyB3ZXJlIGFsc28gcGFzc2VkIHRvIHRoZSBwaWxldXAgZnVuY3Rpb25hbGl0eSBvZiBzYW10b29scwogICAgYW5kIHRoZSB2Y2YvYmNmIHV0aWxpdGllcyBpbiBvcmRlciB0byBtYWtlIGEgbWF0cml4IG9mIGFsbCBvYnNlcnZlZAogICAgZGlmZmVyZW5jZXMgYmV0d2VlbiBlYWNoIHNhbXBsZSB3aXRoIHJlc3BlY3QgdG8gdGhlIHJlZmVyZW5jZS4KNS4gIFRoZSBmcmVlYmF5ZXMgdmFyaWFudCBlc3RpbWF0aW9uIHRvb2wgd2FzIHVzZWQgaW4gYWRkaXRpb24gdG8gIzQKICAgIHRvIHNlYXJjaCBmb3IgdmFyaWFudCBwb3NpdGlvbnMgaW4gYSBtb3JlIHJvYnVzdCBmYXNoaW9uLgo2LiAgVGhlIHRyaW1tZWQgcmVhZHMgd2VyZSBwYXNzZWQgdG8ga3Jha2VuMiB1c2luZyBhIHZpcmFsIGRhdGFiYXNlIGluCiAgICBvcmRlciB0byBsb29rIGZvciBzYW1wbGVzIHdpdGggcG90ZW50aWFsIExSViBzZXF1ZW5jZS4KNy4gIEFuIGV4cGxpY2l0LCBncmVwLWJhc2VkIHNlYXJjaCBmb3Igc3BsaWNlZCBsZWFkZXIgcmVhZHMgd2FzIHVzZWQKICAgIGFnYWluc3QgYWxsIGh1bWFuLWRlcml2ZWQgc2FtcGxlcy4gIFRoZSByZXN1bHRzIGZyb20gdGhpcyB3ZXJlCiAgICBjb3B5L3Bhc3RlZCBpbnRvIHRoZSBzYW1wbGUgc2hlZXQuCgojIyBNdWx0aXBsZSBkYXRhc2V0cwoKSW4gYSBjb3VwbGUgb2YgaW1wb3J0YW50IHdheXMgdGhlIFRNUkMyIGRhdGEgaXMgbXVjaCBtb3JlIGNvbXBsZXggdGhhbiB0aGUKVE1SQzM6CgoxLiAgSXQgY29tcHJpc2VzIG11bHRpcGxlLCBjb21wbGV0ZWx5IHNlcGFyYXRlIHF1ZXJpZXM6CiAgICBhLiAgU2VxdWVuY2luZyB0aGUgcGFyYXNpdGUgc2FtcGxlcwogICAgYi4gIFNlcXVlbmNpbmcgYSBzZXQgb2YgaHVtYW4gbWFjcm9waGFnZSBzYW1wbGVzIHdoaWNoIHdlcmUgaW5mZWN0ZWQKICAgICAgICB3aXRoIHNwZWNpZmljIHBhcmFzaXRlIHNhbXBsZXMuCjIuICBUaGUgcGFyYXNpdGUgdHJhbnNjcmlwdG9taWMgc2FtcGxlcyBjb21wcmlzZSBtdWx0aXBsZSBkaWZmZXJlbnQKICAgIHR5cGVzIG9mIHF1ZXJpZXM6CiAgICBhLiAgRGlmZmVyZW50aWFsIGV4cHJlc3Npb24gdG8gbG9vayBhdCBzdHJhaW4sIHN1c2NlcHRpYmlsaXR5LCBhbmQKICAgIGNsaW5pY2FsIG91dGNvbWVzLgogICAgYi4gIEluZGl2aWR1YWwgdmFyaWFudCBzZWFyY2hlcyB0byBsb29rIGZvciBwb3RlbnRpYWxseSB1c2VmdWwKICAgIFNOUHMgZm9yIGNsYXNzaWZpY2F0aW9uIG9mIHBhcmFzaXRlIHNhbXBsZXMuCjMuICBUaGUgaHVtYW4gbWFjcm9waGFnZSBzYW1wbGVzIG1heSBiZSB1c2VkIHRvIHF1ZXJ5IGJvdGggdGhlIGhvc3QKICAgIGFuZCBwYXJhc2l0ZSB0cmFuc2NyaXB0b21lcyBiZWNhdXNlIChhdCBsZWFzdCB3aGVuIG5vdCBkcnVnCiAgICB0cmVhdGVkKSB0aGVyZSBpcyBhIHRyZW1lbmRvdXMgcG9wdWxhdGlvbiBvZiBwYXJhc2l0ZSByZWFkcyBpbgogICAgdGhlbS4KCiMjIFNhbXBsZSBzaGVldChzKQoKT3VyIHNoYXJlZCBvbmxpbmUgc2FtcGxlIHNoZWV0IGlzIG5lYXJseSBzdGF0aWMgYXQgdGhlIHRpbWUgb2YgdGhpcwp3cml0aW5nICgyMDIyMDkpLCBJIGV4cGVjdCBhdCB0aGlzIHBvaW50IHRoZSBvbmx5IGxpa2VseSB1cGRhdGVzIHdpbGwKYmUgdG8gYW5ub3RhdGUgc29tZSBzdHJhaW5zIGFzIG1vcmUgb3IgbGVzcyBzdXNjZXB0aWJsZSB0byBkcnVnCnRyZWF0bWVudC4KCmBgYHtyfQpzYW1wbGVfc2hlZXQgPC0gInNhbXBsZV9zaGVldHMvbWFjcm9waGFnZV9zYW1wbGVzLnhsc3giCmBgYAoKIyBBbm5vdGF0aW9ucwoKRXZlcnl0aGluZyB3aGljaCBmb2xsb3dzIGRlcGVuZHMgb24gdGhlIEV4aXN0aW5nIFRyaVRyeXBEQiBhbm5vdGF0aW9ucyByZXZpc2lvbgo0NiwgY2lyY2EgMjAxOS4gIFRoZSBmb2xsb3dpbmcgYmxvY2sgbG9hZHMgYSBkYXRhYmFzZSBvZiB0aGVzZSBhbm5vdGF0aW9ucyBhbmQKdHVybnMgaXQgaW50byBhIG1hdHJpeCB3aGVyZSB0aGUgcm93cyBhcmUgZ2VuZXMgYW5kIGNvbHVtbnMgYXJlIGFsbCB0aGUKYW5ub3RhdGlvbiB0eXBlcyBwcm92aWRlZCBieSBUcmlUcnlwREIuCgpUaGUgc2FtZSBkYXRhYmFzZSB3YXMgdXNlZCB0byBjcmVhdGUgYSBtYXRyaXggb2Ygb3J0aG9sb2dvdXMgZ2VuZXMgYmV0d2VlbgpMLnBhbmFtZW5zaXMgYW5kIGFsbCBvZiB0aGUgb3RoZXIgc3BlY2llcyBpbiB0aGUgVHJpVHJ5cERCLgoKVGhlIHNhbWUgZGF0YWJhc2Ugb2YgYW5ub3RhdGlvbnMgYWxzbyBwcm92aWRlcyBtYXBwaW5ncyB0byB0aGUgc2V0IG9mCmFubm90YXRlZCBHTyBjYXRlZ29yaWVzIGZvciB0aGUgTC5wYW5hbWVuc2lzIGdlbm9tZSBhbG9uZyB3aXRoIGdlbmUKbGVuZ3Rocy4KCmBgYHtyLCBldmFsPUZBTFNFfQptZXRhIDwtIGRvd25sb2FkX2V1cGF0aF9tZXRhZGF0YSh3ZWJzZXJ2aWNlID0gInRyaXRyeXBkYiIsIG92ZXJ3cml0ZSA9IEZBTFNFKQpwYW5hbWVuc2lzX2VudHJ5IDwtIGdldF9ldXBhdGhfZW50cnkoIk1IT00iLCBtZXRhZGF0YSA9IG1ldGFbWyJ2YWxpZCJdXSkKcGFuYW1lbnNpc19kYiA8LSBtYWtlX2V1cGF0aF9vcmdkYihwYW5hbWVuc2lzX2VudHJ5KQpwYW5hbWVuc2lzX3BrZyA8LSBwYW5hbWVuc2lzX2RiW1sicGtnbmFtZSJdXQpwYWNrYWdlX25hbWUgPC0gcGFuYW1lbnNpc19kYltbInBrZ25hbWUiXV0KaWYgKGlzLm51bGwocGFuYW1lbnNpc19wa2cpKSB7CiAgcGFuYW1lbnNpc19wa2cgPC0gcGFuYW1lbnNpc19lbnRyeVtbIk9yZ2RiUGtnIl1dCiAgcGFja2FnZV9uYW1lIDwtIHBhbmFtZW5zaXNfcGtnCn0KCnR0IDwtIGxpYnJhcnkocGFuYW1lbnNpc19wa2csIGNoYXJhY3Rlci5vbmx5ID0gVFJVRSkKcGFuYW1lbnNpc19lbnYgPC0gZ2V0MChwYW5hbWVuc2lzX3BrZykKYWxsX2ZpZWxkcyA8LSBjb2x1bW5zKHBhbmFtZW5zaXNfZW52KQphbGxfbHBfYW5ub3QgPC0gc20obG9hZF9vcmdkYl9hbm5vdGF0aW9ucygKICAgIHBhbmFtZW5zaXNfZW52LAogICAga2V5dHlwZSA9ICJnaWQiLAogICAgZmllbGRzID0gYygiYW5ub3RfZ2VuZV9lbnRyZXpfaWQiLCAiYW5ub3RfZ2VuZV9uYW1lIiwKICAgICAgICAgICAgICAgImFubm90X3N0cmFuZCIsICJhbm5vdF9jaHJvbW9zb21lIiwgImFubm90X2Nkc19sZW5ndGgiLAogICAgICAgICAgICAgICAiYW5ub3RfZ2VuZV9wcm9kdWN0IikpKSRnZW5lcwojIyBUZXN0aW5nIHRvIHNlZSBqdXN0IGhvdyBiaWcgdGhlIGZ1bGwgZGF0YWJhc2UgaXMuCiMjIHRlc3RpbmcgPC0gbG9hZF9vcmdkYl9hbm5vdGF0aW9ucyhwYW5hbWVuc2lzX3BrZywga2V5dHlwZSA9ICJnaWQiLCBmaWVsZHMgPSAiYWxsIikKCmxwX2dvIDwtIGxvYWRfb3JnZGJfZ28ocGFuYW1lbnNpc19wa2cpCmxwX2dvIDwtIGxwX2dvWywgYygiR0lEIiwgIkdPIildCmxwX2xlbmd0aHMgPC0gYWxsX2xwX2Fubm90WywgYygiZ2lkIiwgImFubm90X2Nkc19sZW5ndGgiKV0KY29sbmFtZXMobHBfbGVuZ3RocykgIDwtIGMoIklEIiwgImxlbmd0aCIpCmFsbF9scF9hbm5vdFtbImFubm90X2dlbmVfcHJvZHVjdCJdXSA8LSB0b2xvd2VyKGFsbF9scF9hbm5vdFtbImFubm90X2dlbmVfcHJvZHVjdCJdXSkKb3J0aG9zIDwtIHNtKGV4dHJhY3RfZXVwYXRoX29ydGhvbG9ncyhkYiA9IHBhbmFtZW5zaXNfcGtnKSkKZGF0YV9zdHJ1Y3R1cmVzIDwtIGMoZGF0YV9zdHJ1Y3R1cmVzLCAibHBfbGVuZ3RocyIsICJscF9nbyIsICJhbGxfbHBfYW5ub3QiKQpgYGAKCmBgYHtyfQphbGxfaW5zdGFsbGVkIDwtIHJvd25hbWVzKGluc3RhbGxlZC5wYWNrYWdlcygpKQpjYW5kaWRhdGVzIDwtIGdyZXBsKHBhdHRlcm4gPSAiXm9yZy5McGFuYW1lbnNpcy5NSE9NLip2NjguZWcuZGIiLCB4ID0gYWxsX2luc3RhbGxlZCkKZG9fcGFyYXNpdGUgPC0gRkFMU0UKaWYgKHN1bShjYW5kaWRhdGVzKSA+IDApIHsKICBkb19wYXJhc2l0ZSA8LSBUUlVFCiAgb3JnZGJfcGtnX25hbWUgPC0gYWxsX2luc3RhbGxlZFtjYW5kaWRhdGVzXQogIHR0IDwtIGxpYnJhcnkob3JnZGJfcGtnX25hbWUsIGNoYXJhY3Rlci5vbmx5ID0gVFJVRSkKICBwYW5hbWVuc2lzX3BrZyA8LSBnZXQwKG9yZ2RiX3BrZ19uYW1lKQogIGFsbF9maWVsZHMgPC0gY29sdW1ucyhwYW5hbWVuc2lzX3BrZykKICBhbGxfbHBfYW5ub3QgPC0gc20obG9hZF9vcmdkYl9hbm5vdGF0aW9ucygKICAgIHBhbmFtZW5zaXNfcGtnLAogICAga2V5dHlwZSA9ICJnaWQiLAogICAgZmllbGRzID0gYygiYW5ub3RfZ2VuZV9lbnRyZXpfaWQiLCAiYW5ub3RfZ2VuZV9uYW1lIiwKICAgICAgICAgICAgICAgImFubm90X3N0cmFuZCIsICJhbm5vdF9jaHJvbW9zb21lIiwgImFubm90X2Nkc19sZW5ndGgiLAogICAgICAgICAgICAgICAiYW5ub3RfZ2VuZV9wcm9kdWN0IikpKSRnZW5lcwoKICBscF9nbyA8LSBsb2FkX29yZ2RiX2dvKHBhbmFtZW5zaXNfcGtnKQogIGxwX2dvIDwtIGxwX2dvWywgYygiR0lEIiwgIkdPIildCiAgbHBfbGVuZ3RocyA8LSBhbGxfbHBfYW5ub3RbLCBjKCJnaWQiLCAiYW5ub3RfY2RzX2xlbmd0aCIpXQogIGNvbG5hbWVzKGxwX2xlbmd0aHMpICA8LSBjKCJJRCIsICJsZW5ndGgiKQogIGFsbF9scF9hbm5vdFtbImFubm90X2dlbmVfcHJvZHVjdCJdXSA8LSB0b2xvd2VyKGFsbF9scF9hbm5vdFtbImFubm90X2dlbmVfcHJvZHVjdCJdXSkKICBkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJscF9sZW5ndGhzIiwgImxwX2dvIiwgImFsbF9scF9hbm5vdCIpCn0KYGBgCgojIExvYWQgYSBnZW5vbWUKClRoZSBmb2xsb3dpbmcgYmxvY2sgbG9hZHMgdGhlIGZ1bGwgZ2Vub21lIHNlcXVlbmNlIGZvciBwYW5hbWVuc2lzLiAgV2UKbWF5IHVzZSB0aGlzIGxhdGVyIHRvIGF0dGVtcHQgdG8gZXN0aW1hdGUgUENSIHByaW1lcnMgdG8gZGlzY2VybiBzdHJhaW5zLgoKSSBhbSBub3Qgc3VyZSBob3cgdG8gaW5jcmVhc2UgdGhlIG51bWJlciBvZiBvcGVuIGZpbGVzIGluIGEgY29udGFpbmVyLAphcyBhIHJlc3VsdCB0aGlzIGRvZXMgbm90IHdvcmsuCgpgYGB7ciBnZW5vbWUsIGV2YWw9RkFMU0V9CnRlc3RpbmdfcGFuYW1lbnNpcyA8LSBtYWtlX2V1cGF0aF9ic2dlbm9tZShlbnRyeSA9IHBhbmFtZW5zaXNfZW50cnkpCmxpYnJhcnkoYXMuY2hhcmFjdGVyKHRlc3RpbmdfcGFuYW1lbnNpcyksIGNoYXJhY3Rlci5vbmx5ID0gVFJVRSkKbHBfZ2Vub21lIDwtIGdldDAoYXMuY2hhcmFjdGVyKHRlc3RpbmdfcGFuYW1lbnNpcykpCmRhdGFfc3RydWN0dXJlcyA8LSBjKGRhdGFfc3RydWN0dXJlcywgImxwX2dlbm9tZSIpCmBgYAoKIyBHZW5lcmF0ZSBFeHByZXNzaW9uc2V0cyBhbmQgU2FtcGxlIEVzdGltYXRpb24KClRoZSBwcm9jZXNzIG9mIHNhbXBsZSBlc3RpbWF0aW9uIHRha2VzIHR3byBwcmltYXJ5IGlucHV0czoKCjEuICBUaGUgc2FtcGxlIHNoZWV0LCB3aGljaCBjb250YWlucyBhbGwgdGhlIG1ldGFkYXRhIHdlIGN1cnJlbnRseSBoYXZlIG9uIGhhbmQsCiAgICBpbmNsdWRpbmcgZmlsZW5hbWVzIGZvciB0aGUgb3V0cHV0cyBvZiAjMyBhbmQgIzQgYWJvdmUuCjIuICBUaGUgZ2VuZSBhbm5vdGF0aW9ucy4KCkFuIGV4cHJlc3Npb25TZXQob3Igc3VtbWFyaXplZEV4cGVyaW1lbnQpIGlzIGEgZGF0YSBzdHJ1Y3R1cmUgdXNlZCBpbgpSIHRvIGV4YW1pbmUgUk5BU2VxIGRhdGEuICBJdCBpcyBjb21wcmlzZWQgb2YgYW5ub3RhdGlvbnMsIG1ldGFkYXRhLAphbmQgZXhwcmVzc2lvbiBkYXRhLiAgSW4gdGhlIGNhc2Ugb2Ygb3VyIHByb2Nlc3NpbmcgcGlwZWxpbmUsIHRoZQpsb2NhdGlvbiBvZiB0aGUgZXhwcmVzc2lvbiBkYXRhIGlzIHByb3ZpZGVkIGJ5IHRoZSBmaWxlbmFtZXMgaW4gdGhlIG1ldGFkYXRhLgoKIyMgRGVmaW5lIGNvbG9ycwoKVGhlIGZvbGxvd2luZyBsaXN0IGNvbnRhaW5zIHRoZSBjb2xvcnMgd2UgaGF2ZSBjaG9zZW4gdG8gdXNlIHdoZW4KcGxvdHRpbmcgdGhlIHZhcmlvdXMgd2F5cyBvZiBkaXNjZXJuaW5nIHRoZSBkYXRhLgoKYGBge3J9CmNvbG9yX2Nob2ljZXMgPC0gbGlzdCgKICAic3RyYWluIiA9IGxpc3QoCiAgICAjIyAiejEuMCIgPSAiIzMzMzMzMyIsICMjIENoYW5nZWQgdGhpcyB0byAnYnJheicgdG8gbWFrZSBpdCBlYXNpZXIgdG8gZmluZCB0aGVtLgogICAgInoyLjAiID0gIiM1NTU1NTUiLAogICAgInozLjAiID0gIiM3Nzc3NzciLAogICAgInoyLjEiID0gIiM4NzQ0MDAiLAogICAgInoyLjIiID0gIiMwMDAwY2MiLAogICAgInoyLjMiID0gIiNjYzAwMDAiLAogICAgInoyLjQiID0gIiNkZjcwMDAiLAogICAgInozLjIiID0gIiM4ODg4ODgiLAogICAgInoxLjAiID0gIiNjYzAwY2MiLAogICAgInoxLjUiID0gIiNjYzAwY2MiLAogICAgImIyOTA0IiA9ICIjY2MwMGNjIiwKICAgICJ1bmtub3duIiA9ICIjY2JjYmNiIiksCiAgIyMgIm51bGwiID0gIiMwMDAwMDAiKSwKICAienltbyIgPSBsaXN0KAogICAgIm5vbmUiID0gIiMwMDAwMDAiLAogICAgInoyMiIgPSAiIzAwMDBjYyIsCiAgICAiejIzIiA9ICIjY2MwMDAwIiksCiAgImNmIiA9IGxpc3QoCiAgICAiY3VyZSIgPSAiIzAwNmYwMCIsCiAgICAiZmFpbCIgPSAiIzlkZmZhMCIsCiAgICAidW5rbm93biIgPSAiI2NiY2JjYiIsCiAgICAibm90YXBwbGljYWJsZSIgPSAiIzAwMDAwMCIpLAogICJjb25kaXRpb24iID0gbGlzdCgKICAgICJpbmYiID0gIiMxOTljNzUiLAogICAgImluZl9zYiIgPSAiI2Q2NWQwMCIsCiAgICAidW5pbmYiID0gIiM2ZTZlYTMiLAogICAgInVuaW5mX3NiIiA9ICIjZDgzOTU2IiksCiAgInNpZ25pZmljYW5jZSIgPSBsaXN0KAogICAgImx0MCIgPSAiI2ZmZTBlMCIsCiAgICAibHQxIiA9ICIjZmZhMGEwIiwKICAgICJsdDIiID0gIiNmOTQwNDAiLAogICAgImx0NCIgPSAiI2EwMDAwMCIsCiAgICAiZ3QwIiA9ICIjZWVjY2Y5IiwKICAgICJndDEiID0gIiNkZThiZjkiLAogICAgImd0MiIgPSAiI2FkMDdlMyIsCiAgICAiZ3Q0IiA9ICIjNDEwMjU3IiksCiAgImRydWciID0gbGlzdCgKICAgICJub25lIiA9ICIjOTg5ODk4IiwKICAgICJhbnRpbW9ueSIgPSAiIzA4OGI2NCIpLAogICJvbGRuZXciID0gbGlzdCgKICAgICJwcmV2aW91cyIgPSAiIzIyMzNhYSIsCiAgICAiY3VycmVudCIgPSAgIiM5YzAzMDMiKSwKICAiaW5mZWN0ZWRwIiA9IGxpc3QoCiAgICAidW5pbmZlY3RlZCIgPSAiIzY3Njc2NyIsCiAgICAiaW5mZWN0ZWQiID0gIiNhYzA2ZTIiKSwKICAidHJlYXRtZW50X3p5bW8iID0gbGlzdCgKICAgICJpbmZfc2JfejIzIiA9ICIjRTcyOThBIiwKICAgICJpbmZfejIzIiA9ICIjRDk1RjAyIiwKICAgICJ1bmluZl9ub25lIiA9ICIjNjZBNjFFIiwKICAgICJ1bmluZl9zYl9ub25lIiA9ICIjRTZBQjAyIiwKICAgICJpbmZfejIyIiA9ICIjMUI5RTc3IiwKICAgICJpbmZfc2JfejIyIiA9ICIjNzU3MEIzIiksCiAgImRvbm9yIiA9IGxpc3QoCiAgICAiZDAxIiA9ICIjOGQwMDAwIiwKICAgICJkMDIiID0gIiNFNkFCMDIiLAogICAgImQwOSIgPSAiIzc1NzBCMyIsCiAgICAiZDgxIiA9ICIjMjIzM2FhIiksCiAgInN1c2NlcHRpYmlsaXR5IiA9IGxpc3QoCiAgICAicmVzaXN0YW50IiA9ICIjODU2M2E3IiwKICAgICJzZW5zaXRpdmUiID0gIiM4ZDAwMDAiLAogICAgImFtYmlndW91cyIgPSAiI2NiY2JjYiIsCiAgICAidW5rbm93biIgPSAiIzU1NTU1NSIpKQpkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJjb2xvcl9jaG9pY2VzIikKYGBgCgojIE1hY3JvcGhhZ2UgZGF0YQoKQWxsIG9mIHRoZSBhYm92ZSBmb2N1c2VkIGVudGlyZSBvbiB0aGUgcGFyYXNpdGUgc2FtcGxlcywgbm93IGxldCB1cwpwdWxsIHVwIHRoZSBtYWNyb3BoYWdlIGluZmVjdGVkIHNhbXBsZXMuICBUaGlzIHdpbGwgY29tcHJpc2UgdHdvCmRhdGFzZXRzLCBvbmUgb2YgdGhlIGh1bWFuIGFuZCBvbmUgb2YgdGhlIHBhcmFzaXRlLgoKIyMgTWFjcm9waGFnZSBob3N0IGRhdGEKClRoZSBtZXRhZGF0YSBmb3IgdGhlIG1hY3JvcGhhZ2Ugc2FtcGxlcyBjb250YWlucyBhIGNvdXBsZSBvZiBjb2x1bW5zCmZvciBtYXBwZWQgaHVtYW4gYW5kIHBhcmFzaXRlIHJlYWRzLiAgV2Ugd2lsbCB0aGVyZWZvcmUgdXNlIHRoZW0Kc2VwYXJhdGVseSB0byBjcmVhdGUgdHdvIGV4cHJlc3Npb25zZXRzLCBvbmUgZm9yIGVhY2ggc3BlY2llcy4KCioqIE5vdGUgKio6IEkgZm9yZ290IHRvIGNvbW1pdCB0aGUgYWRkaXRpb24gb2YgcGxvdF9tZXRhZGF0YSBmYWN0b3JzKCkgaW4gdGhlIGxhc3QgcnVuIG9mIHRoaXMuCkluIGFkZGl0aW9uLCBJIG5lZWQgdG8gYWRkIGFuIGV4cGxpY2l0IG1vbnRoIHRvCmxvYWRfYmlvbWFydF9hbm5vdGF0aW9ucygpIF9vcl8gY2hhbmdlIHRoZSBmdW5jdGlvbiB0byBzZWFyY2ggYSBjb3VwbGUKbW9yZSBtb250aHMgYmVmb3JlIGl0IHN0b3BzIHRyeWluZyB0byBmaW5kIGFuIGFyY2hpdmUuCgpgYGB7cn0KaHNfYW5ub3QgPC0gbG9hZF9iaW9tYXJ0X2Fubm90YXRpb25zKHllYXIgPSAiMjAyMCIsIG1vbnRoID0gIjA0Iiwgc3ltYm9sX2NvbHVtbnMgPSAiaGduY19zeW1ib2wiKQpoc19hbm5vdCA8LSBoc19hbm5vdFtbImFubm90YXRpb24iXV0KaHNfYW5ub3RbWyJ0cmFuc2NyaXB0Il1dIDwtIHBhc3RlMChyb3duYW1lcyhoc19hbm5vdCksICIuIiwgaHNfYW5ub3RbWyJ0cmFuc2NyaXB0X3ZlcnNpb24iXV0pCnJvd25hbWVzKGhzX2Fubm90KSA8LSBtYWtlLm5hbWVzKGhzX2Fubm90W1siZW5zZW1ibF9nZW5lX2lkIl1dLCB1bmlxdWUgPSBUUlVFKQpyb3duYW1lcyhoc19hbm5vdCkgPC0gcGFzdGUwKCJnZW5lOiIsIHJvd25hbWVzKGhzX2Fubm90KSkKdHhfZ2VuZV9tYXAgPC0gaHNfYW5ub3RbLCBjKCJ0cmFuc2NyaXB0IiwgImVuc2VtYmxfZ2VuZV9pZCIpXQoKc2FuaXRpemVfY29sdW1ucyA8LSBjKCJkcnVnIiwgIm1hY3JvcGhhZ2V0cmVhdG1lbnQiLCAibWFjcm9waGFnZXp5bW9kZW1lIikKbWFjcl9hbm5vdCA8LSBoc19hbm5vdApyb3duYW1lcyhtYWNyX2Fubm90KSA8LSBnc3ViKHggPSByb3duYW1lcyhtYWNyX2Fubm90KSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYXR0ZXJuID0gIl5nZW5lOiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVwbGFjZW1lbnQgPSAiIikKCmhzX21hY3JvcGhhZ2UgPC0gY3JlYXRlX3NlKHNhbXBsZV9zaGVldCwgZ2VuZV9pbmZvID0gbWFjcl9hbm5vdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgZmlsZV9jb2x1bW4gPSAiaGczODEwMGhpc2F0ZmlsZSIpICU+JQogIHNldF9jb25kaXRpb25zKGZhY3QgPSAibWFjcm9waGFnZXRyZWF0bWVudCIpICU+JQogIHNldF9iYXRjaGVzKGZhY3QgPSAibWFjcm9waGFnZXp5bW9kZW1lIikgJT4lCiAgc2FuaXRpemVfbWV0YWRhdGEoY29sdW1ucyA9IHNhbml0aXplX2NvbHVtbnMpICU+JQogIHN1YnNldF9zZShub256ZXJvID0gMTIwMDApCmZpeGVkX2dlbmVuYW1lcyA8LSBnc3ViKHggPSByb3duYW1lcyhhc3NheShoc19tYWNyb3BoYWdlKSksIHBhdHRlcm4gPSAiXmdlbmU6IiwKICAgICAgICAgICAgICAgICAgICAgICAgcmVwbGFjZW1lbnQgPSAiIikKaHNfbWFjcm9waGFnZSA8LSBzZXRfZ2VuZW5hbWVzKGhzX21hY3JvcGhhZ2UsIGlkcyA9IGZpeGVkX2dlbmVuYW1lcykKdGFibGUoY29sRGF0YShoc19tYWNyb3BoYWdlKSRjb25kaXRpb24pCiMjIEFkZGVkIHRvIG1ha2UgYSBzaW1wbGlmaWVkIFBDQSBwbG90Lgpjb2xEYXRhKGhzX21hY3JvcGhhZ2UpW1siZXhwZXJpbWVudCJdXSA8LSAibWFjcm9waGFnZSIKCiMjIFRoZSBmb2xsb3dpbmcgMyBsaW5lcyB3ZXJlIGNvcHkvcGFzdGVkIHRvIGRhdGFzdHJ1Y3R1cmVzIGFuZCBzaG91bGQgYmUgcmVtb3ZlZCBzb29uLgpub3N0cmFpbiA8LSBpcy5uYShjb2xEYXRhKGhzX21hY3JvcGhhZ2UpW1sic3RyYWluaWQiXV0pCmNvbERhdGEoaHNfbWFjcm9waGFnZSlbbm9zdHJhaW4sICJzdHJhaW5pZCJdIDwtICJub25lIgoKY29sRGF0YShoc19tYWNyb3BoYWdlKVtbInN0cmFpbl96eW1vIl1dIDwtIHBhc3RlMCgicyIsIGNvbERhdGEoaHNfbWFjcm9waGFnZSlbWyJzdHJhaW5pZCJdXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIl8iLCBjb2xEYXRhKGhzX21hY3JvcGhhZ2UpW1sibWFjcm9waGFnZXp5bW9kZW1lIl1dKQp1bmluZmVjdGVkIDwtIGNvbERhdGEoaHNfbWFjcm9waGFnZSlbWyJzdHJhaW5fenltbyJdXSA9PSAic25vbmVfbm9uZSIKY29sRGF0YShoc19tYWNyb3BoYWdlKVt1bmluZmVjdGVkLCAic3RyYWluX3p5bW8iXSA8LSAidW5pbmZlY3RlZCIKCmNvbERhdGEoaHNfbWFjcm9waGFnZSlbWyJpbmZlY3RlZHAiXV0gPC0gImluZmVjdGVkIgpjb2xEYXRhKGhzX21hY3JvcGhhZ2UpW3VuaW5mZWN0ZWQsICJpbmZlY3RlZHAiXSA8LSAidW5pbmZlY3RlZCIKCmRhdGFfc3RydWN0dXJlcyA8LSBjKGRhdGFfc3RydWN0dXJlcywgImhzX21hY3JvcGhhZ2UiKQpgYGAKCiMjIERvdWJsZS1jaGVjayBzYW1wbGUgSURzIGFnYWluc3QgdGhlIHNhbXBsZSBzaGVldAoKMSBzYW1wbGUgaGFzIGJlZW4gZXhjbHVkZWQgZnJvbSB0aGUgYW5hbHlzaXMgYnV0IGlzIGluIHRoZSBzYW1wbGUKc2hlZXQuICBJIGFtIHJlYXNvbmFibHkgY2VydGFpbiBJIGtub3cgd2hpY2gsIGJ1dCB3aWxsIGRvdWJsZS1jaGVjawpoZXJlLgoKYGBge3IsIGV2YWw9RkFMU0V9CnNhbXBsZV9zaGVldF9pZHMgPSBjKCJUTVJDMzAwNTEiLCJUTVJDMzAwNTciLCJUTVJDMzAwNTkiLCJUTVJDMzAwNjAiLCJUTVJDMzAwNjEiLCJUTVJDMzAwNjIiLAogICAgICAgICAgICAgICAgICAgICAiVE1SQzMwMDYzIiwiVE1SQzMwMDY0IiwiVE1SQzMwMDY1IiwiVE1SQzMwMDY2IiwiVE1SQzMwMDY3IiwiVE1SQzMwMDY5IiwKICAgICAgICAgICAgICAgICAgICAgIlRNUkMzMDExNyIsIlRNUkMzMDE2MiIsIlRNUkMzMDI0MyIsIlRNUkMzMDI0NCIsIlRNUkMzMDI0NSIsIlRNUkMzMDI0NiIsCiAgICAgICAgICAgICAgICAgICAgICJUTVJDMzAyNDciLCJUTVJDMzAyNDgiLCJUTVJDMzAyNDkiLCJUTVJDMzAyNTAiLCJUTVJDMzAyNTEiLCJUTVJDMzAyNTIiLAogICAgICAgICAgICAgICAgICAgICAiVE1SQzMwMjY2IiwiVE1SQzMwMjY3IiwiVE1SQzMwMjY4IiwiVE1SQzMwMjg2IiwiVE1SQzMwMzI2IiwiVE1SQzMwMzE2IiwKICAgICAgICAgICAgICAgICAgICAgIlRNUkMzMDMxNyIsIlRNUkMzMDMyMiIsIlRNUkMzMDMyMyIsIlRNUkMzMDMyOCIsIlRNUkMzMDMxOCIsIlRNUkMzMDMxOSIsCiAgICAgICAgICAgICAgICAgICAgICJUTVJDMzAzMjQiLCJUTVJDMzAzMjUiLCJUTVJDMzAzMjAiLCJUTVJDMzAzMjEiLCJUTVJDMzAzMjciLCJUTVJDMzAzMTIiLAogICAgICAgICAgICAgICAgICAgICAiVE1SQzMwMjk3IiwiVE1SQzMwMjk4IiwiVE1SQzMwMjk5IiwiVE1SQzMwMzAwIiwiVE1SQzMwMjk1IiwiVE1SQzMwMjk2IiwKICAgICAgICAgICAgICAgICAgICAgIlRNUkMzMDMwMyIsIlRNUkMzMDMwNCIsIlRNUkMzMDMwMSIsIlRNUkMzMDMwMiIsIlRNUkMzMDMxNCIsIlRNUkMzMDMxNSIsCiAgICAgICAgICAgICAgICAgICAgICJUTVJDMzAzMTMiKQpmb3VuZCA8LSBzYW1wbGVfc2hlZXRfaWRzICVpbiUgY29sbmFtZXMoYXNzYXkoaHNfbWFjcm9waGFnZSkpCmNvbG5hbWVzKGFzc2F5KGhzX21hY3JvcGhhZ2UpKVshZm91bmRdCmBgYAoKIyMgU3Vic2V0IGFuZCBjcmVhdGUgZGlmZmVyZW50IGdyb3VwaW5ncwoKYGBge3J9CmFsbF9odW1hbiA8LSBzYW5pdGl6ZV9tZXRhZGF0YShoc19tYWNyb3BoYWdlLCBjb2x1bW5zID0gImRydWciKSAlPiUKICBzZXRfY29uZGl0aW9ucyhmYWN0ID0gImRydWciKSAlPiUKICBzZXRfYmF0Y2hlcyhmYWN0ID0gInR5cGVvZmNlbGxzIikKZGF0YV9zdHJ1Y3R1cmVzIDwtIGMoZGF0YV9zdHJ1Y3R1cmVzLCAiYWxsX2h1bWFuIikKCiMjIFRoZSBmb2xsb3dpbmcgMyBsaW5lcyB3ZXJlIGNvcHkvcGFzdGVkIHRvIGRhdGFzdHJ1Y3R1cmVzIGFuZCBzaG91bGQgYmUgcmVtb3ZlZCBzb29uLgpub19zdHJhaW5faWR4IDwtIGNvbERhdGEoYWxsX2h1bWFuKVtbInN0cmFpbmlkIl1dID09ICJub25lIgojI2NvbERhdGEoYWxsX2h1bWFuKVtbInN0cmFpbmlkIl1dIDwtIHBhc3RlMCgicyIsIGNvbERhdGEoYWxsX2h1bWFuKVtbInN0cmFpbmlkIl1dLAojIyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIl8iLCBjb2xEYXRhKGFsbF9odW1hbilbWyJtYWNyb3BoYWdlenltb2RlbWUiXV0pCmNvbERhdGEoYWxsX2h1bWFuKVtub19zdHJhaW5faWR4LCAic3RyYWluaWQiXSA8LSAibm9uZSIKdGFibGUoY29sRGF0YShhbGxfaHVtYW4pW1sic3RyYWluaWQiXV0pCgphbGxfaHVtYW5fdHlwZXMgPC0gc2V0X2NvbmRpdGlvbnMoYWxsX2h1bWFuLCBmYWN0ID0gInR5cGVvZmNlbGxzIikgJT4lCiAgc2V0X2JhdGNoZXMoZmFjdCA9ICJkcnVnIikKZGF0YV9zdHJ1Y3R1cmVzIDwtIGMoZGF0YV9zdHJ1Y3R1cmVzLCAiYWxsX2h1bWFuX3R5cGVzIikKCnR5cGVfenltb19mYWN0IDwtIHBhc3RlMChjb2xEYXRhKGFsbF9odW1hbl90eXBlcylbWyJjb25kaXRpb24iXV0sICJfIiwKICAgICAgICAgICAgICAgICAgICAgICAgIGNvbERhdGEoYWxsX2h1bWFuX3R5cGVzKVtbIm1hY3JvcGhhZ2V6eW1vZGVtZSJdXSkKdHlwZV96eW1vIDwtIHNldF9jb25kaXRpb25zKGFsbF9odW1hbl90eXBlcywgZmFjdCA9IHR5cGVfenltb19mYWN0KQpkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJ0eXBlX3p5bW8iKQoKdHlwZV9kcnVnX2ZhY3QgPC0gcGFzdGUwKGNvbERhdGEoYWxsX2h1bWFuX3R5cGVzKVtbImNvbmRpdGlvbiJdXSwgIl8iLAogICAgICAgICAgICAgICAgICAgICAgICAgY29sRGF0YShhbGxfaHVtYW5fdHlwZXMpW1siZHJ1ZyJdXSkKdHlwZV9kcnVnIDwtIHNldF9jb25kaXRpb25zKGFsbF9odW1hbl90eXBlcywgZmFjdCA9IHR5cGVfZHJ1Z19mYWN0KQpkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJ0eXBlX2RydWciKQoKc3RyYWluX2ZhY3QgPC0gY29sRGF0YShhbGxfaHVtYW5fdHlwZXMpW1sic3RyYWluaWQiXV0KdGFibGUoc3RyYWluX2ZhY3QpCgpuZXdfY29uZGl0aW9ucyA8LSBwYXN0ZTAoY29sRGF0YShoc19tYWNyb3BoYWdlKVtbIm1hY3JvcGhhZ2V0cmVhdG1lbnQiXV0sICJfIiwKICAgICAgICAgICAgICAgICAgICAgICAgIGNvbERhdGEoaHNfbWFjcm9waGFnZSlbWyJtYWNyb3BoYWdlenltb2RlbWUiXV0pCiMjIE5vdGUgdGhlIHNhbml0aXplKCkgY2FsbCBpcyByZWR1bmRhbnQgd2l0aCB0aGUgYWRkaXRpb24gb2Ygc2FuaXRpemUoKSBpbiB0aGUKIyMgZGF0YXN0cnVjdHVyZXMgZmlsZSwgYnV0IEkgZG9uJ3Qgd2FudCB0byB3YWl0IHRvIHJlcnVuIHRoYXQuCmhzX21hY3IgPC0gc2V0X2NvbmRpdGlvbnMoaHNfbWFjcm9waGFnZSwgZmFjdCA9IG5ld19jb25kaXRpb25zKSAlPiUKICBzYW5pdGl6ZV9tZXRhZGF0YShjb2x1bW4gPSAiZHJ1ZyIpICU+JQogIHNldF9zZV9jb2xvcnMoY29sb3JfY2hvaWNlc1tbInRyZWF0bWVudF96eW1vIl1dKSAlPiUKICBzdWJzZXRfc2Uoc3Vic2V0ID0gInR5cGVvZmNlbGxzIT0nVTkzNyciKQpkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJoc19tYWNyIikKCmdnc3RhdHNfcGFyYXNpdGUgPC0gcGxvdF9tZXRhZGF0YV9mYWN0b3JzKGhzX21hY3IsIGNvbHVtbiA9ICJwYXJhc2l0ZW1hcHBpbmdyYXRlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0eXBlID0gImdnc3RhdHMiLCBzY2FsZSA9ICJsb2cyIikKcHAoZmlsZSA9ICJpbWFnZXMvZ2dzdGF0c19wYXJhc2l0ZXJhdGVfYWxsX21hY3JvcGhhZ2VfZHJ1Z190cmVhdG1lbnQucG5nIikKZ2dzdGF0c19wYXJhc2l0ZQpkZXYub2ZmKCkKZ2dzdGF0c19wYXJhc2l0ZQoKaHNfbWFjcl9kcnVnX3NlIDwtIHNldF9jb25kaXRpb25zKGhzX21hY3IsIGZhY3QgPSAiZHJ1ZyIpCgpoc19tYWNyX3N0cmFpbl9zZSA8LSBzZXRfY29uZGl0aW9ucyhoc19tYWNyLCBmYWN0ID0gIm1hY3JvcGhhZ2V6eW1vZGVtZSIpICU+JQogIHN1YnNldF9zZShzdWJzZXQgPSAibWFjcm9waGFnZXp5bW9kZW1lICE9ICdub25lJyIpCmRhdGFfc3RydWN0dXJlcyA8LSBjKGRhdGFfc3RydWN0dXJlcywgImhzX21hY3Jfc3RyYWluX3NlIikKCnRhYmxlKGNvbERhdGEoaHNfbWFjcilbWyJzdHJhaW5pZCJdXSkKYGBgCgpMZXQgdXMgc2VlIGlmIHRoZSBzYW5rZXkgcGxvdCBvZiB0aGVzZSBzYW1wbGVzIGxvb2tzIHVzZWZ1bC4uLgoKYGBge3J9Cmdnc3RhdHNfc2xyZWFkcyA8LSBwbG90X21ldGFkYXRhX2ZhY3RvcnMoaHNfbWFjcm9waGFnZSwgY29sdW1uID0gImhpc2F0bHBzaW5nbGVtYXBwZWQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHR5cGUgPSAiZ2dzdGF0cyIsIHNjYWxlID0gImxvZzIiKQpwcChmaWxlID0gImltYWdlcy9nZ3N0YXRzX3NscmVhZHNfYWxsX21hY3JvcGhhZ2UucG5nIikKZ2dzdGF0c19zbHJlYWRzCmRldi5vZmYoKQpnZ3N0YXRzX3NscmVhZHMKZ2dzdGF0c192aW9saW4gPC0gcGxvdF9tZXRhZGF0YV9mYWN0b3JzKGhzX21hY3JvcGhhZ2UsIGNvbHVtbiA9ICJoaXNhdGxwc2luZ2xlbWFwcGVkIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNjYWxlID0gImxvZzIiKQpnZ3N0YXRzX3Zpb2xpbgoKbWFjcl9zYW5rZXkgPC0gcGxvdF9tZXRhX3NhbmtleShoc19tYWNyb3BoYWdlLCBjb2xvcl9jaG9pY2VzID0gY29sb3JfY2hvaWNlcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWN0b3JzID0gYygib2xkbmV3IiwgImRydWciLCAiaW5mZWN0ZWRwIiwgIm1hY3JvcGhhZ2V6eW1vZGVtZSIpKQptYWNyX3NhbmtleQpgYGAKCkZpbmFsbHksIHNwbGl0IG9mZiB0aGUgVTkzNyBzYW1wbGVzLgoKYGBge3J9CmhzX3U5MzcgPC0gc3Vic2V0X3NlKGhzX21hY3JvcGhhZ2UsIHN1YnNldCA9ICJ0eXBlb2ZjZWxscyE9J01hY3JvcGhhZ2VzJyIpCmRhdGFfc3RydWN0dXJlcyA8LSBjKGRhdGFfc3RydWN0dXJlcywgImhzX3U5MzciKQpgYGAKCiMjIE1hY3JvcGhhZ2UgcGFyYXNpdGUgZGF0YQoKSW4gdGhlIHByZXZpb3VzIGJsb2NrLCB3ZSB1c2VkIGEgbmV3IGludm9jYXRpb24gb2YgZW5zZW1ibC1kZXJpdmVkCmFubm90YXRpb24gZGF0YSwgdGhpcyB0aW1lIHdlIGNhbiBqdXN0IHVzZSBvdXIgZXhpc3RpbmcgcGFyYXNpdGUgZ2VuZQphbm5vdGF0aW9ucy4KCmBgYHtyfQppZiAoaXNUUlVFKGRvX3BhcmFzaXRlKSkgewogIGxwX21hY3JvcGhhZ2UgPC0gY3JlYXRlX3NlKAogICAgc2FtcGxlX3NoZWV0LCBmaWxlX2NvbHVtbiA9ICJscGFuYW1lbnNpc3YzNmhpc2F0ZmlsZSIsIGdlbmVfaW5mbyA9IGFsbF9scF9hbm5vdCwKICAgIHNhdmVmaWxlID0gZ2x1ZSgicmRhL2xwX21hY3JvcGhhZ2Utdnt2ZXJ9LnJkYSIpLAogICAgYW5ub3RhdGlvbiA9ICJvcmcuTHBhbmFtZW5zaXMuTUhPTUNPTDgxTDEzLnY0Ni5lZy5kYiIpICU+JQogICAgc2V0X2NvbmRpdGlvbnMoZmFjdCA9ICJtYWNyb3BoYWdlenltb2RlbWUiKSAlPiUKICAgIHNldF9iYXRjaGVzKGZhY3QgPSAibWFjcm9waGFnZXRyZWF0bWVudCIpCgogIHVuZmlsdF93cml0dGVuIDwtIHdyaXRlX3NlKAogICAgbHBfbWFjcm9waGFnZSwKICAgIGV4Y2VsID0gZ2x1ZSgiYW5hbHlzZXMvbWFjcm9waGFnZV9kZS97dmVyfS9yZWFkX2NvdW50cy9scF9tYWNyb3BoYWdlX3JlYWRzX3VuZmlsdGVyZWQtdnt2ZXJ9Lnhsc3giKSkKCiAgbHBfbWFjcm9waGFnZV9maWx0IDwtIHN1YnNldF9zZShscF9tYWNyb3BoYWdlLCBub256ZXJvID0gMjUwMCkgJT4lCiAgICBzZW1hbnRpY19maWx0ZXIoc2VtYW50aWMgPSBjKCJhbWFzdGluIiwgImdwNjMiLCAibGVpc2htYW5vbHlzaW4iKSwKICAgICAgICAgICAgICAgICAgICBzZW1hbnRpY19jb2x1bW4gPSAiYW5ub3RfZ2VuZV9wcm9kdWN0IikKICBkYXRhX3N0cnVjdHVyZXMgPC0gYyhkYXRhX3N0cnVjdHVyZXMsICJscF9tYWNyb3BoYWdlIiwgImxwX21hY3JvcGhhZ2VfZmlsdCIpCgogIGZpbHRfd3JpdHRlbiA8LSB3cml0ZV9zZShscF9tYWNyb3BoYWdlX2ZpbHQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGV4Y2VsID0gZ2x1ZSgiYW5hbHlzZXMvbWFjcm9waGFnZV9kZS97dmVyfS9yZWFkX2NvdW50cy9scF9tYWNyb3BoYWdlX3JlYWRzX2ZpbHRlcmVkLXZ7dmVyfS54bHN4IikpCiAgbHBfbWFjcm9waGFnZSA8LSBscF9tYWNyb3BoYWdlX2ZpbHQKICBscF9tYWNyb3BoYWdlX25vc2IgPC0gc3Vic2V0X3NlKGxwX21hY3JvcGhhZ2UsIHN1YnNldD0iYmF0Y2ghPSdpbmZfc2InIikKICBscF9ub3NiX3dyaXRlIDwtIHdyaXRlX3NlKAogICAgbHBfbWFjcm9waGFnZV9ub3NiLAogICAgZXhjZWwgPSBnbHVlKCJhbmFseXNlcy9tYWNyb3BoYWdlX2RlL3t2ZXJ9L3JlYWRfY291bnRzL2xwX21hY3JvcGhhZ2Vfbm9zYl9yZWFkcy12e3Zlcn0ueGxzeCIpKQogIGRhdGFfc3RydWN0dXJlcyA8LSBjKGRhdGFfc3RydWN0dXJlcywgImxwX21hY3JvcGhhZ2Vfbm9zYiIpCgogIHNwZWMgPC0gbWFrZV9ybmFzZXFfc3BlYygpCiAgdGVzdCA8LSBnYXRoZXJfcHJlcHJvY2Vzc2luZ19tZXRhZGF0YShzYW1wbGVfc2hlZXQsIHNwZWNpZmljYXRpb24gPSBzcGVjKQp9CmBgYAoKIyBTYXZlIGFsbCBkYXRhIHN0cnVjdHVyZXMgaW50byBvbmUgcmRhCgpgYGB7cn0KZm91bmRfaWR4IDwtIGRhdGFfc3RydWN0dXJlcyAlaW4lIGxzKCkKaWYgKHN1bSghZm91bmRfaWR4KSA+IDApIHsKICBub3RfZm91bmQgPC0gZGF0YV9zdHJ1Y3R1cmVzWyFmb3VuZF9pZHhdCiAgd2FybmluZygiU29tZSBkYXRhc3RydWN0dXJlcyB3ZXJlIG5vdCBnZW5lcmF0ZWQ6ICIsIHRvU3RyaW5nKG5vdF9mb3VuZCksICIuIikKICBkYXRhX3N0cnVjdHVyZXMgPC0gZGF0YV9zdHJ1Y3R1cmVzW2ZvdW5kX2lkeF0KfQpzYXZlKGxpc3QgPSBkYXRhX3N0cnVjdHVyZXMsIGZpbGUgPSBnbHVlKCJyZGEvdG1yYzJfZGF0YV9zdHJ1Y3R1cmVzLXZ7dmVyfS5yZGEiKSkKYGBgCgpgYGB7cn0KcGFuZGVyOjpwYW5kZXIoc2Vzc2lvbkluZm8oKSkKbWVzc2FnZSgiVGhpcyBpcyBocGdsdG9vbHMgY29tbWl0OiAiLCBnZXRfZ2l0X2NvbW1pdCgpKQptZXNzYWdlKCJTYXZpbmcgdG8gIiwgc2F2ZWZpbGUpCiMgdG1wIDwtIHNtKHNhdmVtZShmaWxlbmFtZSA9IHNhdmVmaWxlKSkKYGBgCgpgYGB7ciBsb2FkbWVfYWZ0ZXIsIGV2YWw9RkFMU0V9CnRtcCA8LSBsb2FkbWUoZmlsZW5hbWUgPSBzYXZlZmlsZSkKYGBgCg==