Kotlin — Copy (big) files

Recently I found myself in the situation of coping files of (potentially) any size, so I started in a simple way.

The simple way means just reading the bytes of the file and writing them into a new one, so starting with the file to copy we could simply do something like this:

Pretty simple indeed, let’s take a closer look to what we are doing here:

  • Getting an output stream from the destination file (where we are going to write the bytes from the original file).
  • Reading the bytes to copy from the original file.
  • Writing them in the output stream we previously opened and finally closing that stream.

All good so far. Now let’s focus on the documentation of the readBytes() function:

Gets the entire content of this file as a byte array.This method is not recommended on huge files. It has an internal limitation of 2 GB byte array size.

As the documentation states, this is not suitable for files bigger than 2 GB.

Copying bigger files

How can we avoid the 2Gb limitation?

We can read the bytes ourself using a buffer: we can read chunks of bytes, store them in a buffer and then append those chunks one by one to the destination file.

In that way we will avoid 2 issues:

  • Reading all the bytes of a big file at once, exceeding the API limitations
  • Creating a huge ByteArray which will also exceed API limitations

A sample of how this is implemented can be found at https://github.com/Alexs784/Kotlin-Copy-file-sample

Particularly interesting is the testing part, where :

  • Create files filled with random bytes and then copy them
  • Generate a checksum for the original and the copied files to verify that the bytes were copied properly.

The creation of the files was also tricky since we cannot just write x bytes on a file: if the size is quite big, we will have troubles there as well.
So the approach was similar to when copying files — we use a buffer and we write a small amount of megabytes at each iteration:

More about this can be found in the linked repository, specifically in the FileUtil class and the FileUtilTest.




Senior Android Engineer @WeTransfer 🤖 🇻🇪🇮🇹 Paraglider, Runner, Kayaker.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Open Source Windows UPS Monitor and Shutdown utility: Watch-Win32_UPS

Easily parallelize very large scale gsutil operations (deletes, copies, rsync’s, moves)

Kubernetes : Assign CPU Resource Defaults and Limits to Containers

How Dubbo-Based Microservices Are Used in Insurance Companies

34. To Encapsulate Or Not To Encap

appdeploy: a helm chart for consistency

Top 5 Myths about Agile Framework Project Management

Why Have Alibaba Started to Embrace OAM When Serverless Is So Popular?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alessandro Mautone

Alessandro Mautone

Senior Android Engineer @WeTransfer 🤖 🇻🇪🇮🇹 Paraglider, Runner, Kayaker.

More from Medium

Support for Split Application Binary (.OBB)

Firebase Sign In Android + Saving user’s data in Realtime Database + Saving user’s session

Jetpack Compose Basics

Comparing ready-made solutions for implementing the MVI architecture on Android