In April 2024 FileSender celebrated its 15th anniversary. It officially started 15 years ago as a joint project of the NRENs from Norway, Australia and Ireland. Today FileSender is deployed as a national R&E service in more than 45 countries and has been selected for the file transfer functionality of the EU Node of the European Open Science Cloud (EOSC). We looked back on previous years and summarised the key steps of the development, faced challenges and achievements the FileSender community is proud of.
FileSender aims to be a widely deployed service enabling anyone to easily and securely transfer files of any size from any person or machine to any other person or machine. FileSender explicitly targets mass usage and is built to service the 99% of users who would rather spend their time on other things than figuring out how to perform large file transfers.
The initiative began in 2007 after discussions in TERENA, back then the European NREN collaboration organisation, showed a clear need for a service to transfer large files.
Initially, the FileSender value proposition was: painless sharing of arbitrarily large files with a private audience through a trusted intermediary. As security requirements on all R&E services increased over the years, this matured into ‘secure and painless sharing’.
“While it started with the basic value proposition of easy transfer of large files, e.g. not having to send USB sticks, hard drives, etc., over time we’ve seen a shift towards the notion of ‘secure transfer’ becoming more important, which we facilitate through both ‘the NREN as a trusted intermediary’ and through the browser-native end-to-end encryption of transfers”.
Timeline
Key steps during the period 2009-2024 include:
- April 2009 – Project started as a self-financed open source collaboration to produce a file transfer application with web-only delivery. Lead developer Chris Richter.
- November 2009 – First national R&E service deployed by AARNet, the Australian NREN.
- December 2009 – First external code security audit conducted to support the security by design principle. Annual security audits have been conducted since.
- January 2011 – FileSender 1.0 released, based on Adobe Flash with Google Gears browser plug-in for large file transfer support. It was well tested to be deployed and used with very low support.
- November 2011 – FileSender 1.1 released with large file transfer support moved to bleeding edge HTML5 File API.
- March 2013 – FileSender 1.5 released. User interface moved from Adobe Flash to native PHP+JavaScript. This release became the main workhorse for service delivery until the release of version 2.0.
- 2014 – Development of multi-file upload started. Major re-implementation of many components. Transition to new lead developer Étienne Méléard (in-kind contribution from RENATER, the French NREN). Start of multi-language support.
- December 2016 – FileSender was established as the first programme in newly set-up the Commons Conservancy foundation, which provides a permanent and independent legal-administrative home, making long-term governance and financing easier. Transition to the current lead developer Ben Martin.
- June 2018 – FileSender 2.0 released for production use with encrypted transfer. This release marked the change from manual to automated testing with a CI/CD pipeline.
- March 2020 – FileSender brand new website and style were created and launched. A well-structured website, with a trendy design for the FileSender logo and eye-catching visuals, supports the brand and thus the service.
- November 2023 – Beta-release 3.0 with the redesigned and refreshed FileSender user interface (UI) and user experience (UX), improving its look and feel. It went through 3 phases: research and usage data collecting; developing a wireframe prototype based on the renewed user journey and user feedback; and creating a design system for FileSender.
One of the key steps was choosing a web-only delivery model early on. At the time this was a quite novel approach, as back then research software was typically required to be installed on the user’s desktop. FileSender has been browser-native from the start: no client install is needed. We believe this was and remains an essential choice to ensure large-scale uptake and ease of use at a low cost point, even though it requires effort to keep functionality up-to-date as browsers change.
Having a reliable lead developer and maintainer, contributing structurally to FileSender, is crucial to ensure steady development and integration of community contributions over time. Currently we engage with the FileSender 3rd lead developer, and the project is still going strong. His principal tasks encompass watching the mailing list and GitHub issues, creating resolutions for problems as they arise, considering improvements to functionality, efficiency and code design, as triaging which new item to implement and ensuring long term sound technical direction.
It was very important to include encrypted transfers. Initially, it was a ‘forked version’, with a proof of concept version created in 2012 at SURF, the Dutch NREN, as a response to the corresponding demand from the Amsterdam University Medical Center. After many years of availability as a beta-release, field testing and security audits of the encryption module, the functionality officially became available to a wide audience when it was included in the FileSender 2.0 release of June 2018.
Translation management (with the support of the POEditor translation management platform) makes it possible to render the FileSender user interface in a lot of languages. This provides an opportunity for the community to contribute more easily and collaborate, enabling crowd-sourcing of translations without requiring significant management overhead. That was an explicit decision very early on to fully leverage collaborative development services, spending our effort on progressing FileSender rather than on the support infrastructure.
Over the last 15 years many NRENs and individual people contributed funds and time in large and small amounts to FileSender development demonstrating the value of producing software together: a better result for a lower cost per contributing organisation and a rewarding environment for people to work in.
“Aves IT performs a variety of services, one of which is application management for FileSender installations, for NRENs and other business customers. We offer solutions for running FileSender in your own datacentre/cloud or in our datacentre. Aves IT contributes in-kind and donates time to help FileSender grow and mature”.
Challenges
With FileSender’s increasing popularity, aligning opinions on the direction of FileSender takes more time and patience. It also means the quality of the software becomes more and more important, a challenge emphasised by the increased focus on security, usability and ease of provisioning of services. This has an impact on the budget which needs to steadily increase as well.
That brings us to another challenge: keeping contributions (financial or in-kind) coming in over time, to ensure FileSender development doesn’t stop. ‘Software rot’ occurs quite quickly and a development (and maintenance) pause of a year is not a healthy situation. The need for a smooth flow of financial contributions ended up spawning the Commons Conservancy foundation. Through it we have the capability to receive funds from many contributors with low effort from our side. The Commons Conservancy also ensures longevity of governance and a smooth running collaboration ‘substrate’. We are in a good place regarding our financial-administrative organisation, but it took years to bring all of this together in a ‘happy flow’.
One of the major difficulties in the long term is supporting the popular browsers and making sure appropriate testing of the FileSender software is in place. Browser technology, its standards and deployment are unpredictable. We’ve seen Internet Explorer come and disappear, we’ve seen Web 3.0 appear. Under these conditions, it’s inevitable to manage long protracted painful rewrites, UX changes and code refactoring – and survive it! Many projects succumb to ‘the second-system effect’. Also, the file encryption support and other functionality, e.g. virtual zip creation, wants to use the latest features in browsers to offer the best security and experience, but not all browsers support all the desired functionality. There were difficulties in the past supporting older web browsers such as Internet Explorer 11 which had a tendency to remain in use in e.g. conservatively managed hospital desktop environments.
Achievements
Our steady pace of development and adoption has led FileSender to become the de-facto standard for web-based large file transfer in R&E. Appearing as a requirement in the EOSC EU Node specifications, then bidding and winning the bid as a Managed Large File Transfer service for end-users, the FileSender service becomes a key component of the emerging EOSC Federation.
From the technical point of view we know the following features are highly appreciated:
- No file size limit other than the limit configured by a FileSender service provider and the patience of the user;
- Browser-native encryption, privacy by design and focus on secure software;
- API for integration in automated research workflows and larger portals;
- Automatic upload resume even when the IP address of the machine has changed, which allows robust upload on poor networks and permits a person to move a laptop to another location and resume the upload;
- Externalised authentication using federated login by design;
- Direct creation of zip archives for non-encrypted content on download.
We managed to turn the FileSender project into an actual community with many organisations, users and developers – all contributing to support FileSender over time, not just two funded developers and a lot of lurkers. At present, the FileSender community comprises enthusiasts both within and outside the R&E organisations worldwide.
“I am proud of the FileSender community, which acts with respect to each other, but at the same time is not afraid to challenge each other. People work together internationally, across time zones to try to make FileSender a better product. In the portfolio of SURF, this is one of the products that users rate highly and one of the key services our institutions rely on”.
During the last three years, FileSender became a part of the annual NREN Satisfaction Survey, organised among GÉANT’s members. The overall picture of the results is very positive:
- 2021: 10 NRENs rated the quality of the FileSender software as ‘very high’, 13 NRENs – as ‘high’, 14 NRENs – ‘no opinion’ or did not answer the question (37 responses in total);
- 2022: 11 NRENs – ‘very high’, 12 NRENs – ‘high’, 10 NRENs – ‘no opinion’ or did not answer the question (33 responses in total);
- 2023: 10 NRENs – ‘very high’, 14 NRENs – ‘high’, 12 NRENs – ‘no opinion’ or did not answer the question (36 responses in total).
While there are various online service offerings in the file-sharing market, FileSender is one of very few long term stable open source packages that allows you to provide your own large file transfer service. It remains popular due to its embrace of the open source paradigm in spirit and code, privacy by design, end-to-end encryption support and above all native browser support for sending very large files.
We have come quite far with this collective effort during these 15 years, but if we are to continue this – we’ll have to scale up the effort (and hence the budget) to support increased quality and security requirements. That’s why we call upon all NRENs to join the R&E FileSender infrastructure and start running a FileSender also for your community! It is available for download on GitHub: https://github.com/filesender/filesender/releases/.
Please see Support and Mailing lists to stay updated on the latest FileSender developments and important news.
