phyNGSC is a hybrid strategy between MPI and OpenMP to accelerate the compression of big FASTQ datasets by combining the best features of distributed and shared memory architectures to balance the load of work among processes, alleviate memory latency by exploiting locality and accelerate I/O by reducing excessive read/write operations and inter-node message exchange. The algorithm introduces a novel timestamp-based approach which allows concurrent writing of compressed data in a non-deterministic order and thereby allows us to exploit a high amount of parallelism. As a proof-of-concept, we implemented some methods developed for DSRC v1 to underline the compression portion of our hybrid parallel strategy, since it exhibits superior performance for sequential solutions. The parallel algorithm is developed using C/C++, MPI and OPENMP constructs
Status
The method development for this work is complete