10:00 am – 5:10 pm, 8:00 pm – 11:45 pm
Parallelizing image analysis
issues with parfor loops
- much faster
- easily exploading memory
- observation: launch and shutdown of parpool is not fast — takes several minutes even to spin down.
- Found issue: using matrices xDim x yDim x numGenes for indexing is is too memory intensive for 1001 genes and parallel pool of 20 cells.
- no global variables allowed inside parfor loops
success
- now running efficiently through all data.
- uses up to 20 GB memory and 65% CPU
- runs rather slower on 1000 gene data than 140 gene data: loop over genes are slowing things down a bit.
- slow step is still the intial warp and deconvolution — this can run pretty efficiently overnight though.
- Morgan may reboot itself before my code finishes though…
Chromatin paper
- edits back from XZ. Make changes for tomorrow.