DataNumen Disk Image is a powerful tool to clone and restore disks or drives. It can create and restore the disk image or drive image byte by byte. Useful for backup, data recovery, disk/drive copy & cloning, and forensic.
Super Stick Recovery Tool V1 0 2 19 Zip
I generally recurse when I can detect layering, but - you might need to run this tool more than once since many images have multiple layers of encapsulation:morpheus@Zephyr (/Downloads) % imjtool 8WCN25WW/cap/qcfirmware8998v1.0.1075.2507.cap extract UEFI firmware image detected at offset 0xa15Size: 2ded80, tag: 4856465f, attr: 3feff, checksum:512d, version: 2, blockSize: 0x40, blockCount:0xb7b6Next GUID@0xa5d: Lenovo container (2de814 bytes, type Raw, attr 40)Extracting Lenovo containerNext GUID@0x2df275: C7340E65-0D5D-43D6-ABB7-39751D5EC8E7 (510 bytes, type Raw, attr 40)Extracting C7340E65-0D5D-43D6-ABB7-39751D5EC8E7# # And again:#morpheus@Zephyr (/Downloads) % imjtool extracted/Lenovo\ container UEFI firmware image detected at offset 0x7eba0Warning: non zeros in headerSize: 200000, tag: 4856465f, attr: cfeff, checksum:c8a7, version: 2, blockSize: 0x200, blockCount:0x1000Warning: additional content at offset 0x27eba0Next GUID@0x7ebe8: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF (fa0 bytes, type Padding, attr 0)Next GUID@0x7fb88: QCOM XBL BOOT.XF Terse Executable (26018 bytes, type Security Core, attr 28)Section @0x7fba0 Type: Raw, Size: 0xf34 (empty) Section @0x80ad4 Type: Terse Executble, Size: 0x250cc Next GUID@0xa5ba0: uefiplat.cfg (24ab bytes, type Freeform, attr 0)Section @0xa5bb8 Type: UI, Size: 0x1e uefiplat.cfgSection @0xa5bd8 Type: Raw, Size: 0x2473 Next GUID@0xa8050: QCOM package (1adcd0 bytes, type Firmware Volume Image, attr 0)COMPRESSED - EE4E5898-3914-4259-9D6E-DC7BD79403CF - Magic 0x5d@0xa8080LZMA! 1adca0 bytesexamining decompressed data (8542152 bytes)Size: 8257c0, tag: 4856465f, attr: 3feff, checksum:948d, version: 2, blockSize: 0x40, blockCount:0x2095fWarning: additional content at offset 0x8257c0Next GUID@0x48: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF (2c bytes, type Padding, attr 0)Next GUID@0x78: FC510EE7-FFDC-11D4-BD41-0080C73C8881 (54c bytes, type Freeform, attr 0)Section @0x90 Type: Raw, Size: 0x534 Next GUID@0x5c8: D6A2CB7F-6A18-4E2F-B43B-9920A733700A (27030 bytes, type Driver eXecution Environment Core, attr 0)Section @0x5e0 Type: PE32, Size: 0x27004 Section @0x275e4 Type: UI, Size: 0x14 DxeCoreNext GUID@0x275f8: ARM CPU DXE (c03c bytes, type Driver, attr 0)Section @0x27610 Type: Driver eXecution Env Dependencies, Size: 0x6 Section @0x27618 Type: PE32, Size: 0xc004 Section @0x3361c Type: UI, Size: 0x18 ArmCpuDxeNext GUID@0x33638: Runtime DXE (803e bytes, type Driver, attr 0)Section @0x33650 Type: Driver eXecution Env Dependencies, Size: 0x6 Section @0x33658 Type: PE32, Size: 0x8004 Section @0x3b65c Type: UI, Size: 0x1a RuntimeDxeI don't purport to cover all EFI GUIDs here. Your favorite tool is probably better. I built this for my own use cases (primarily, command line, greppable, scriptable, cross platorm), and I think it's useful enough to provide freely. If your specific image isn't supported, you can always drop me a line. Ranting on twitter denigrating my work and/or me won't help. And btw, you're welcome.Version 1.2 ChangesSupports super.img images (liblp logical partitions)
root@Qilin (/NewAndroidBook/ddb/S20Ultra/...)# imjtool super.img extract Warning: super.img is likely truncated or still compressedSparse image v1.0 detected, 2304000 blocks of 4096 bytes2304000 blocks of 4096 bytes compressed into 56 chunks (20% compressed)Extracted image is in extracted/image.imgroot@Qilin (/NewAndroidBook/ddb/S20Ultra/...)# imjtool extracted/image.img extractliblp dynamic partition (super.img) - Blocksize 0x1000, 2 slotsLP MD Header @0x3000, version 10.0, with 4 logical partitions on block device at partition super, first sector: 0x800 Partitions @0x3080 in 2 groups: Group 0: default Group 1: group_basic Name: system (read-only, spanning 1 extents and 5597 MB) - extracted Name: vendor (read-only, spanning 1 extents and 1091 MB) - extracted Name: product (read-only, spanning 1 extents and 676 MB) - extracted Name: odm (read-only, spanning 1 extents and 4 MB) - extractedroot@Qilin (/NewAndroidBook/ddb/S20Ultra/...)# file extracted/*extracted/image.img: dataextracted/odm.img: Linux rev 1.0 ext2 filesystem data, UUID=79c8b8f8-84be-5f6f-b675-dd53f53ebb2c, volume name "odm" (extents) (large files) (huge files)extracted/product.img: Linux rev 1.0 ext2 filesystem data, UUID=8bf42db5-b3bc-5890-8b2a-8eeeba73b344, volume name "product" (extents) (large files) (huge files)extracted/system.img: Linux rev 1.0 ext2 filesystem data, UUID=2939f12c-6689-53a4-b94a-1c1ce1c83fbd (extents) (large files) (huge files)extracted/vendor.img: Linux rev 1.0 ext2 filesystem data, UUID=a8b0696d-9320-55bc-b768-8579563bdfdc, volume name "vendor" (extents) (large files) (huge files)Supports DTBO
Supports Samsung TOC
Supports Brotli compressed (....new.br) block based images. For now, work with me and supply the number of blocks by getting them from the ...transfer.list (second line):
root@Qilin (/NewAndroidBook/ddb/RedmiK30) # head -2 vendor.transfer.list4345427root@Qilin (/NewAndroidBook/ddb/RedmiK30) #BLOCKS=345427 imjtool vendor.new.dat.br \ stl=vendor.transfer.listAttempting Brotli decompression. You might need to supply NUMBLOCKS=(2nd line on the list) while J works this feature.Image written to /tmp/extracted.imgroot@Qilin (/NewAndroidBook/ddb/RedmiK30) # file /tmp/extracted.img/tmp/extracted.img: Linux rev 1.0 ext2 filesystem data, UUID=2b96c597-1e2f-5ee1-9851-c4a9fa9de36e,volume name "vendor" (extents) (large files) (huge files)v2.0(β) - ZIP/XZ/LZ4/BZ2 supportIncluding partial zips! For example:
zpaq is a free and open sourceincremental, journaling command-line archiver for Windows, Linux and Mac OS/X.Incremental means that when you back up your hard drive, forexample: zpaq add e:\backup.zpaq c:\*then only those files whose last-modified date or size has changed sincethe previousbackup are added. For 100 GB of files, this typically takes about a minute, vs.an hour to create the first version.Journaling means that the archive is append-only. When you addfiles or directories to the archive, both the old and new versions are saved.You can recover old versions by specifying the date or version number,for example: zpaq extract e:\backup.zpaq c:\Users\Bob -to tmp -until 2013-10-30will extract all the files and directories in c:\Users\Bob as of the lastbackup on or before Oct. 30, 2013 and put them in a directory named tmp.zpaq is faster and compresses better than most other popular archiversand backup programs,especially for realistic backups that have a lot of duplicate filesand a lot of already compressed files.Archive size vs. time tocompress and extract 10 GB (79,431 files) to an external USB harddrive at default and maximumsettings on a Dell Latitude E6510 laptop (Core i7 M620, 2+2hyperthreads, 2.66 GHz, 4 GB, Ubuntu Linux, Wine 1.6).Data from10 GB Benchmark (system 4).Feature comparison zpaq pcompress exdupe freearc obnam rar 7zip zipWindows W W W W W WLinux L L L L L L L LUpdate U U U U U U UIncremental I I I I IRollback R RDedupe D D D DEncryption E E E E E EGUI G G G GFree F F F F F F FOpen source O O O O O O OSpecification S S Downloadzpaq.exe for Windows.The latest version is zpaq v7.15,released Aug. 17, 2016. The downloadcontain source code (zpaq.cpp, libzpaq.cpp, libzpaq.h), Windows executables(32 or 64 bit, XP or later), documentation (zpaq.pod),and a Makefile for compiling in Linux, BSD, or Mac OS/X.You may need unzip.exe to unzip from theWindows command line.zpaq man page (HTML, latest version).The ZPAQ archive format is described by aspecification and reference decoder.A test caseexercising all of the specification features should decompress to theCalgary corpus.The compression algorithm is describedhere.The source code includes the libzpaq API providing compressionand decompression services for applications in C++.Developers may be interested in thezpaqd development tool and sample configuration filesfound on the utilities page.zpaq is written by Matt Mahoney and released to the public domain.It includes code from libdivsufsort 2.0 (C) Yuta Mori, 2003-2008, MIT license,public domain code for AES from libtomcrypt by Tom St Denis andpublic domain code for salsa20 by D. J. Bernstein.FeaturesA zpaq archive can contain at most 4 billion files and at most250 terabytes of data after deduplication and before compression.zpaq is for user-level backups. Do not use it to back up the operatingsystem or any software that requires a password to install.zpaq saves regular files and directories, last-modified dates (to thenearest second), and (optionally) Windows attributes or Linuxpermissions. It does not follow or save symbolic links or junctions.It unknowingly follows hard links. It does not save owner or group IDs,ACLs, extended attributes,the registry, or special file types like devices, sockets, or named pipes.Open standard specificationThe zpaq archive format is described by a precisespecification and reference decoder (above).The format is not encumbered by any patents or pending patentsin any country as far as I know. I have purposely published allpast versions (below) to establish prior art so that no patents can be filed.Backward and forward compatibilityAll versions of zpaq can read archives produced by older versionsback to version 1.00 (March 2009). To some extent, older versions canread archives produced by newer versions (forward compatibility) providedthey don't use any unsupported features. These are as follows:v1.00 (Mar. 2009). Level 1 format. Streaming archives with at leastone context model. Does not support deduplication or rollback.v5.00 (Aug. 2012). Level 2 format. Adds support for compressionwith pre/post processing with no context modeling (e.g. uncompressed or LZ77).v6.00 (Sept. 2012). Journaling format (dedupe and rollback).v6.44 (Jan. 2014). Encrypted archives.v6.47 (Jan. 2014). Multi-part archives. Older versions can read them if concatenated.Many intermediate versions include compression improvements. This doesnot break forward compatibility because the decompression codeis stored in the archive. The code is written in a sandboxed,virtual machine language called ZPAQL. On x86-32 and x86-64processors, the ZPAQL code is translated to machine code and executed,so it is as fast as compression algorithms written in compiledlanguages like C or C++. On other hardware, the ZPAQL code is interpreted,which takes about twice as long.For example, the following will create a streaming archive using BWTcompression that can be extracted by all versions back to v1.00, even thoughmost of these versions could not compress using BWT. zpaq add archive.zpaq files -method s4.3ci1RollbackAn archive is updated only by appending changes to it. You canroll back the archive to an earlier state by using the -until optionto specify the date and time or version number where to stop reading.When updating, -until will truncate the archive at thatpoint before appending. So if you backed up some files you didn't meanto, then you can truncate the last update and repeat: zpaq add backup c:\ -not c:\tmp -until -1Transacted updatesUpdates are committed by first appending a temporary header and thenupdating it when all of the compressed data and index changes are appended.If you interrupt zpaq (by typing Ctrl-C), then the partially appendeddata will be ignored and overwritten on the next update.DeduplicationWhen adding files,zpaq uses a rolling hash function to split files into fragmentswith an average size of 64 KB along content-dependent boundaries.Then it computes the SHA-1 hash of the fragment and compares it withsaved hashes from the current and previous versions.If it finds a match then the fragment is not stored.Deduplication requires 1 MB of memory per GB of deduplicatedbut uncompressed archive data to update, and 0.5 MB per GB tolist or extract.Incremental update and restoreFiles are added only if the date has changed since the last update.You can use the -force option to override, but in this case the filewill be deduplicated and not saved unless the contents have really changed.This is slower than comparing dates but faster than compressing it again.Extraction will not clobber existing files unless you give the-force option to allow overwrite. In this case, the file to be overwrittenis compared with the stored hashes and not decompressed unless the sizeor contents is different.Remote archive supportzpaq updates an archive by appending changes to it. To supportremote backups without having to move huge files,zpaq can put the appended changes into a separate,numbered file that you would copy or move to remote storage.You can concatenate the parts to form a complete archive, or simplyread them all at once by specifying a pattern in the archive namelike "part???.zpaq".zpaq will then search for part001.zpaq, part002.zpaq, etc. and regardthe concatenated sequence as a single archive.To make incremental backups with a local copy: zpaq add "arc???" files (copy arc001.zpaq) zpaq add "arc???" files (copy arc002.zpaq) zpaq list "arc???" (show contents) zpaq extract "arc???" (restore)To back up without keeping a local copy of the archive, you keepa small local index (arc000.zpaq) as a copy of the remote archive minusthe compressed file contents. zpaq maintains consistency between theindex and archive. zpaq add "arc???" files -index arc000.zpaq (move arc001.zpaq) zpaq add "arc???" files -index arc000.zpaq (move arc002.zpaq) zpaq list arc000 (show arc???.zpaq contents)EncryptionArchives can be encrypted using AES-256 in CTR mode.A password must be given every time an encrypted archive is used.Keys are strengthened with Scrypt(N=16384, r=8, p=1)(requiring 208M operations and 16 MB memory)to slow down brute force search for weakkeys. Encrypted archives are prefixed with a 32 byte random salt, whichalso provides an 8 byte IV for the first half of the 16 byte AES counter.If a remote archive has a local index, then both are encrypted with thesame key but different salts to generate independent keystreams.Encryption provides privacy but not authentication against tampering.All of the encryption code (AES, Scrypt, SHA-1, SHA-256)is public domain and tested against published test vectors.The AES code is derived from libtomcrypt 1.17.Multithreaded compressionzpaq has 5 compression levels. The default, -method 1, is the fastest.It is best for backups where you compress often and extract rarely.-method 2 compresses slower but decompresses as fast as -method 1.It is best for distributing files where you compress once and extractoften. Methods 3, 4, and 5 are slower with better compression.Fragments not removed by deduplication are packed into blocksfor compression. Files are sorted by filename extension and then bydecreasing size in order to group similar files together.The block size is 16 MB for method 1 and 64 MB for higher methods.You can change the block size to trade compression for memory usage.Blocks are compressed or decompressed in parallel in separate threads.zpaq automatically detects the number of processor cores and uses all of themin the 64 bit version or at most 2 in the 32 bit version (which is limitedto 2 GB memory).You can use the -threads option to change the number of threads.Resident memory per thread required to compress or decompress isapproximately as follows. Virtual memory usage may be higher. Method Compress Decompress Algorithm ------ -------- ---------- --------- 1 128 MB 32 MB LZ77 2 450 MB 128 MB LZ77 3 450 MB 400 MB LZ77+CM or BWT 4 550 MB 550 MB LZ77+CM, BWT or CM 5 850 MB 850 MB CMMethod 1 uses LZ77, compressing by replacing duplicate strings withpointers to previous occurrences. Method 2 is the same but spendsmore time looking for better matches (using a suffix array instead ofa hash table). Method 3 uses either BWT (context sorting) orLZ77 for long matches and an order 1 context model and arithmeticcoding for literals depending on the file type. Method 4 either uses LZ77, BWTor a high order context model. Method 5 uses a complex, high order contextmixing model with over 20 bit prediction components.All methods except 5 test whether the data appears to be compressibleor already compressed (random). Uncompressible data is simply stored.An E8E9 filter is applied if x86 data (normally found in .exe and .dllfiles) is detected. The filter replaces x86 CALL and JMP relativeaddresses with absolute addresses to make the data more compressible.Data analysiszpaq has list options to make it easier to examine the contentsof archives containing millions of files. For example, the followingcompares external dir1 to internal dir2 and lists only differences.Files are compared quickly by size and last modified date, or thoroughlyby reading the file, computing its SHA-1 hashes and comparing withthe hashes stored in the archive. zpaq list backup dir1 -to dir2 -not = (compare dates) zpaq list backup dir1 -to dir2 -not = -force (compare contents)Other useful list options: -only *.exe List only files ending with .exe -not *.exe Don't list files matching a pattern. -summary 20 List the 20 largest files and identify duplicates. -all Show all file versions. -until 20 List contents as of the 20'th updateError detection and recoveryzpaq archives are designed to minimize data loss if damaged. Anarchive is divided into blocks that can be decompressed independently.Each block begins with a 13 byte tag that can be found by scanningif the previous block is damaged. Each block ends with the SHA-1 hashof the uncompressed data, which is verified to detect errors. Blockswith hash mismatches or other errors are ignored with a warningwithout killing zpaq.Each update contains 4 types of blocks.C - Update header: date, size of compressed data.D - Compressed data fragments, list of fragment sizes.H - List of fragment hashes and sizes, one per D block.I - Index updates: list of files updated or deleted. Each update includes the date, attributes, and list of fragments.C blocks are used to skip over D blocks to read the index quickly.They are not needed to extract.If a D or H block is lost then so are any files that point to it.If an I block is lost, then so areany files in it. I blocks are small (16 KB) to minimize damage.When extracting files, the D block is decompressed up to the lastused fragment and those fragments are hashed and compared to thestored hashes in the H block.The zpaq -test -all extract option will decompress internallyand verify all of the fragment hashes without writing the output files.Public Domain APIThe source download includes libzpaq,a public domain application programming interface (API) in C++ thatprovides streaming compression and decompression servicesto and from files, strings, or arrays using built-in and customcompression algorithms. To use the code, you include libzpaq.hin your program and link to libzpaq.cpp.The API documentation is in libzpaq.h. The precise semantics isdescribed in the ZPAQ specification.In the simplest case,the application provides an error handling function and derivedimplementations of two abstract classes, Reader and Writer,specifying the input and output byte streams. For example, to compressfrom stdin to stdout (assuming binary I/O as in Linux): #include "libzpaq.h" #include #include void libzpaq::error(const char* msg) // print message and exit fprintf(stderr, "Oops: %s\n", msg); exit(1); class In: public libzpaq::Reader public: int get() return getchar(); // returns byte 0..255 or -1 at EOF in; class Out: public libzpaq::Writer public: void put(int c) putchar(c); // writes 1 byte 0..255 out; int main() libzpaq::compress(&in, &out, "1"); // -method 1 To decompress: libzpaq::decompress(&in, &out);There are also functions for reading and writing block and segmentheaders and for passing specialized methods or ZPAQL code to the compressor,as documented in libzpaq.h.The ZPAQ utilities pagecontains sample compression algorithms written in ZPAQLand a tool zpaqd for running, testing, and debugging ZPAQL.HistoryAll versions of the software and documentation can be downloadedbelow. The major development steps were:Feb. 15, 2009: zpaq 0.01, First of 9 experimental, mutually incompatible versions.Mar. 12, 2009: zpaq 1.00. First level 1 standard conforming archiver using interpreted ZPAQL for forward and backward compatibility.Sept. 29, 2010: libzpaq 1.00. First version of API providing compression services to applications in C++.Nov. 5, 2010: libzpaq 2.01. If an external C++ compiler is available then zpaq will translate ZPAQL to C++ and recompile itself to improve speed.Jan. 26, 2011: pzpaq 0.01. First multi-threaded version (later renamed zp, then merged back into zpaq).Nov. 13, 2011: libzpaq/zpaq 4.00. First version with JIT-accelerated ZPAQL for x86/64, eliminating need for external C++ compiler.Feb. 1, 2012: libzpaq 5.00. Level 2 standard allowed high speed compression without a context model (pre/post processing only).Sept. 26, 2012: zpaq 6.00. Journaling format to support deduplication, fast indexing, update recovery, and storing multiple versions of files and directories.June 11, 2013: zpaq 6.27. Moved developer tools into zpaqd.Dec. 20, 2013: zpaq 6.43. Adds AES encryption.Nov. 22, 2014: zpaq 6.56. Supports remote multi-part archives with a local index.zpaq versions 7.00 and older are licensed under GPL v3.The SHA-1 code used in versions prior to libzpaq 1.00is derived fromRFC 3174, whichis copyright (C) 2001, The Internet Society. Please see thisdocument for the full license. 2ff7e9595c
Comments