Kotlin — Copy (big) files
Recently I found myself in the situation of coping files of (potentially) any size, so I started in a simple way.
The simple way means just reading the bytes of the file and writing them into a new one, so starting with the file to copy we could simply do something like this:
Pretty simple indeed, let’s take a closer look to what we are doing here:
- Getting an output stream from the destination file (where we are going to write the bytes from the original file).
- Reading the bytes to copy from the original file.
- Writing them in the output stream we previously opened and finally closing that stream.
All good so far. Now let’s focus on the documentation of the
Gets the entire content of this file as a byte array.This method is not recommended on huge files. It has an internal limitation of 2 GB byte array size.
As the documentation states, this is not suitable for files bigger than 2 GB.
Copying bigger files
How can we avoid the 2Gb limitation?
We can read the bytes ourself using a buffer: we can read chunks of bytes, store them in a buffer and then append those chunks one by one to the destination file.
In that way we will avoid 2 issues:
- Reading all the bytes of a big file at once, exceeding the API limitations
- Creating a huge
ByteArraywhich will also exceed API limitations
A sample of how this is implemented can be found at https://github.com/Alexs784/Kotlin-Copy-file-sample
Particularly interesting is the testing part, where :
- Create files filled with random bytes and then copy them
- Generate a checksum for the original and the copied files to verify that the bytes were copied properly.
The creation of the files was also tricky since we cannot just write x bytes on a file: if the size is quite big, we will have troubles there as well.
So the approach was similar to when copying files — we use a buffer and we write a small amount of megabytes at each iteration: