Package org.apache.commons.compress.archivers.zip

Class ParallelScatterZipCreator

  • public class ParallelScatterZipCreator
extends java.lang.Object
    Creates a zip in parallel by using multiple threadlocal ScatterZipOutputStream instances.

    Note that until 1.18, this class generally made no guarantees about the order of things written to the output file. Things that needed to come in a specific order (manifests, directories) had to be handled by the client of this class, usually by writing these things to the ZipArchiveOutputStream before calling writeTo on this class.

    The client can supply an ExecutorService, but for reasons of memory model consistency, this will be shut down by this class prior to completion.

    Since:
    1.10

    • Constructor Detail

      • ParallelScatterZipCreator

        public ParallelScatterZipCreator()
        Create a ParallelScatterZipCreator with default threads, which is set to the number of available processors, as defined by Runtime.availableProcessors()

      • ParallelScatterZipCreator

        public ParallelScatterZipCreator​(java.util.concurrent.ExecutorService executorService)
        Create a ParallelScatterZipCreator
        Parameters:
        executorService - The executorService to use for parallel scheduling. For technical reasons, this will be shut down by this class.

      • ParallelScatterZipCreator

        public ParallelScatterZipCreator​(java.util.concurrent.ExecutorService executorService,
                                 ScatterGatherBackingStoreSupplier backingStoreSupplier)
        Create a ParallelScatterZipCreator
        Parameters:
        executorService - The executorService to use. For technical reasons, this will be shut down by this class.
        backingStoreSupplier - The supplier of backing store which shall be used

      • ParallelScatterZipCreator

        public ParallelScatterZipCreator​(java.util.concurrent.ExecutorService executorService,
                                 ScatterGatherBackingStoreSupplier backingStoreSupplier,
                                 int compressionLevel)
                          throws java.lang.IllegalArgumentException
        Create a ParallelScatterZipCreator
        Parameters:
        executorService - The executorService to use. For technical reasons, this will be shut down by this class.
        backingStoreSupplier - The supplier of backing store which shall be used
        compressionLevel - The compression level used in compression, this value should be -1(default level) or between 0~9.
        Throws:
        java.lang.IllegalArgumentException - if the compression level is illegal
        Since:
        1.21

    • Method Detail

      • addArchiveEntry

        public void addArchiveEntry​(ZipArchiveEntry zipArchiveEntry,
                            InputStreamSupplier source)
        Adds an archive entry to this archive.

        This method is expected to be called from a single client thread

        Parameters:
        zipArchiveEntry - The entry to add.
        source - The source input stream supplier

      • addArchiveEntry

        public void addArchiveEntry​(ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)
        Adds an archive entry to this archive.

        This method is expected to be called from a single client thread

        Parameters:
        zipArchiveEntryRequestSupplier - Should supply the entry to be added.
        Since:
        1.13

      • submit

        public final void submit​(java.util.concurrent.Callable<? extends java.lang.Object> callable)
        Submit a callable for compression.
        Parameters:
        callable - The callable to run, created by createCallable, possibly wrapped by caller.
        See Also:
        for details of if/when to use this.

      • createCallable

        public final java.util.concurrent.Callable<ScatterZipOutputStream> createCallable​(ZipArchiveEntry zipArchiveEntry,
                                                                                  InputStreamSupplier source)
        Create a callable that will compress the given archive entry.

        This method is expected to be called from a single client thread.

        Consider using addArchiveEntry, which wraps this method and submitStreamAwareCallable. The most common use case for using createCallable and submitStreamAwareCallable from a client is if you want to wrap the callable in something that can be prioritized by the supplied ExecutorService, for instance to process large or slow files first. Since the creation of the ExecutorService is handled by the client, all of this is up to the client.
        Parameters:
        zipArchiveEntry - The entry to add.
        source - The source input stream supplier
        Returns:
        A callable that should subsequently passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.

      • writeTo

        public void writeTo​(ZipArchiveOutputStream targetStream)
             throws java.io.IOException,
                    java.lang.InterruptedException,
                    java.util.concurrent.ExecutionException
        Write the contents this to the target ZipArchiveOutputStream.

        It may be beneficial to write things like directories and manifest files to the targetStream before calling this method.

        Calling this method will shut down the ExecutorService used by this class. If any of the Callables submitted to this instance throws an exception, the archive can not be created properly and this method will throw an exception.

        Parameters:
        targetStream - The ZipArchiveOutputStream to receive the contents of the scatter streams
        Throws:
        java.io.IOException - If writing fails
        java.lang.InterruptedException - If we get interrupted
        java.util.concurrent.ExecutionException - If something happens in the parallel execution

      • getStatisticsMessage

        public ScatterStatistics getStatisticsMessage()
        Returns a message describing the overall statistics of the compression run
        Returns:
        A string