9:30a – 7:45p, 10:00p – 12:15a
Genomics
- Cajal slowing down (2536 of 4385. Though all 4 variants of stringency is 4x 4385)
- CPU cap semi-respected, code is waiting for free cpu dips to launch new jobs. Target is 75%. Running 70 – 98%, averaging around 80%. Though a half dozen copies of Insight3 are running from other users…
- source of problem: min wait is just to launch another job. Doesn’t actually kill the running job. Need to explore windows wmic commands more to get run time and send kill commands when run time is exceeded.
- Moved all active windows to one side of screen. If these windows are still active in 30min – 1 hr, they get killed.
- in fact, still only 11 windows active out of a 16 cap, so the CPU cap is hitting first. Though I guess these ‘timed out’ windows are certainly still eating CPU as well.
- Holyscratch is supposed to work. Attempting to launch all OligoArray commands there.
- All OligoArray jobs on Odyssey stuck in PENDING Purgatory.
- Still running on Monet (942 of 4385).
- Real problem is that blastall likes to hang while chewing up a complete processor. blastall shouldn’t stay in the process list for more than a few seconds judging by the average turnover. Occassionally it sticks, with an unchanging Memory usage and steady processor usage.
- Cajal finally finished r = 1 run.
To check:
- make sure regions BLAT to correct genome region and correct strand
Ph simulations
- increasing bond break rate smooths the cluster distribution curve.
- cluster dynamics largely driven by chromatin folding effects
- Now running in parallel new full scale (I hope) simulations of Diff Binding and Capping models (on Monet).
More Notes for Odyssey:
- A somewhat useful SLURM reference.
- Some minimal complement of information to Harvard’s minimal explanation or the entirely useless SLURM project page
- Another useful reference for SLURM
- cancel all jobs
scancel --state=PENDING --user=bob --partition=debug
Chromatin Colors Project
Chromatin simulations
- All batch 50 jobs ‘running’
- however there are no saved results from my demo with 5 so I think this all failed.
- However, the startup.m from my home directory does execute upon launching matlab, so maybe there’s hope.
- could set up new test script to see if this is true?
- All FAILED or TIMEOUT
- code in fact seems to still have bugs. startup.m used the windows directory paths (
Y:\) not the Odyssey directory paths - started running a few parallel simulations on Tuck (not much memory available though).
- Also very important: Reset the random number generator if doing multiple simulations in multiple version of matlab. Do this no more than once per session.
s = RandStream(‘mt19937ar’,’Seed’,’shuffle’);
RandStream.setGlobalStream(s);
Presentation / Slide prep:
- start putting slides together for meeting with XZ.
- Add G06 images
- Add region coverage plots.
Cell staining
- Day 2 of in situs hybes for short regions (last attempt).
- F08 worked! Nice surprise. Even RNase treated.
- Blocking remaining samples. Will add Dm01-488.
- finished staining and rinsing (if any of these others actually work).
Dot Analysis
- Analyzed D09 data
- F09 still crashes new version of ChromatinCropper
Coding: improve 3D analysis
- To do: New Z-alignment.
- Needs to treat beads as individuals,
- doesn’t assume all beads reach the wx = wy crossing point
- doesn’t assume all beads are in the same plane
- Captured more and denser bead fields on cells.
Cell culture
- flipped one of the more populated small vials that had low (<40%) attachment into new small flask. Now >90% attachment.
General Multithreading code development
- wrote new script ProcessTimeout to find and selective kill process that have been running too long
- Good for purging hung
blastall.execommands. - Requires R2013a. Won’t work on Tuck Cajal at present