Kotlin — Copy (big) files

Recently I found myself in the situation of coping files of (potentially) any size, so I started in a simple way.

The simple way means just reading the bytes of the file and writing them into a new one, so starting with the file to copy we could simply do something like this:

Pretty simple indeed, let’s take a closer look to what we are doing here:

• Getting an output stream from the destination file (where we are going to write the bytes from the original file).
• Reading the bytes to copy from the original file.
• Writing them in the output stream we previously opened and finally closing that stream.

All good so far. Now let’s focus on the documentation of the readBytes() function:

Gets the entire content of this file as a byte array.
This method is not recommended on huge files. It has an internal limitation of 2 GB byte array size.

As the documentation states, this is not suitable for files bigger than 2 GB.

Copying bigger files

How can we avoid the 2Gb limitation?

We can read the bytes ourself using a buffer: we can read chunks of bytes, store them in a buffer and then append those chunks one by one to the destination file.

In that way we will avoid 2 issues:

• Reading all the bytes of a big file at once, exceeding the API limitations
• Creating a huge ByteArray which will also exceed API limitations

A sample of how this is implemented can be found at https://github.com/Alexs784/Kotlin-Copy-file-sample

Particularly interesting is the testing part, where :

• Create files filled with random bytes and then copy them
• Generate a checksum for the original and the copied files to verify that the bytes were copied properly.

The creation of the files was also tricky since we cannot just write x bytes on a file: if the size is quite big, we will have troubles there as well.
So the approach was similar to when copying files — we use a buffer and we write a small amount of megabytes at each iteration:

More about this can be found in the linked repository, specifically in the FileUtil class and the FileUtilTest.