java - TrueZip compression taking too much time -
i using truezip
compression. here code looks
public string compress() throws ioexception { if (loglocations.isempty()) { throw new illegalstateexception("no logs provided compress"); } removedestinationifexists(desiredarchive); final tfile destinationarchive = new tfile(desiredarchive + "/diagnostics"); (final string loglocation : loglocations) { final tfile log = new tfile(loglocation); if (!log.exists()) { logger.debug("{} not exist, ignoring."); continue; } if (log.isdirectory()) { log.cp_r(destinationarchive); } else { final string newloglocation = new tfile(destinationarchive.getabsolutepath()) + slash + getlognamefrompath(loglocation); log.cp(new tfile(newloglocation)); } } return destinationarchive.getenclarchive().getabsolutepath(); }
and test
@test public void testbenchmarkwithhprof() throws ioexception { final filewriter loglocations; string loglocationpath = "/users/harit/downloads/tmp/loglocations.txt"; { loglocations = new filewriter(loglocationpath); loglocations.write("test3"); loglocations.write("\n"); loglocations.close(); } final lplogcompressor compressor = new lplogcompressor("/users/harit/downloads/tmp", new file(loglocationpath), "/users/harit/downloads/tmp/testout"); final long starttime = system.currenttimemillis(); compressor.compress(); system.out.println("time taken (msec): " + (system.currenttimemillis() - starttime)); }
and data directory test3
looks
test3/ java_pid1748.hprof
the file size 2.83gb
when ran test, took on 22 minutes.
when compress same file using native osx compress (right click -> compress)
, takes 2 minutes
why there of difference?
thanks
update
based on @satnam recommendation, attached debugger see whats going on , find
none of truezip threads running? really? apologies using profiler first time
the reason in case using default deflater deflater.best_compression
.
i override zipdriver
class on level
import de.schlichtherle.truezip.fs.archive.zip.zipdriver; import de.schlichtherle.truezip.socket.iopoolprovider; import java.util.zip.deflater; public class overridezipdriver extends zipdriver { public overridezipdriver(final iopoolprovider iopoolprovider) { super(iopoolprovider); } @override public int getlevel() { return deflater.default_compression; } }
and in compressor
class, did
public lplogcompressor(final string logprocessorinstallpath, final file loglocationssource, final string desiredarchive) throws ioexception { this.desiredarchive = desiredarchive + dot + getdatetimestampformat() + zip; loglocations = getloglocations(logprocessorinstallpath, loglocationssource); enablelogcompression(); } private static void enablelogcompression() { tconfig.get().setarchivedetector( new tarchivedetector(tarchivedetector.null, new object[][]{ {"zip", new overridezipdriver(iopoollocator.singleton)},})); tconfig.push(); }
you can read thread here
Comments
Post a Comment