Shares

GNOME Software is the default application in the GNOME desktop environment to manage software. It also allows you to receive firmware updates through an underlaying daemon called “fwupd“, which is based on an platform called “LVFS“.

In order to understand the relationship in a clearer way, you can think of LVFS as the online platform where hardware vendors come and upload new versions of their firmware which will be later available to download via fwupd. GNOME Software utilizes the fwupd daemon in order to download and install these updates. fwupd is a dependency for GNOME Software.

The whole ecosystem is developed mainly by Richard Hughes, who is working currently for Red Hat, and who’s also the original creator of PackageKit. But it’s worthy to mention that Red Hat doesn’t develop/manage the project directly, but rather, contributes to it with financial & logistic support.

The Privacy Concern

fwupd is an integrated part of GNOME Software. In order to be able to receive updates for firmware available in your computer, fwupd retrieves a metadata file from fwupd.org (which is named LVFS) and matches on client-side to see if there’s an update to that firmware or not. If there’s an update, the firmware file will be grabbed and installed from fwupd.org.

Additionally, fwupd (upon each checking for updates in GNOME Software) sends your machine ID (hashed), IP address, current Linux distribution name and version and client user-agent to fwupd.org.

According to fwupd.org/privacy, some parts of this data is being shared by 3rd-party hardware vendors, who may be interested in the number of people downloading these updates and what kind of Linux distros they are using. IP addresses are not shared, but client user-agent & hashed machine IDs are. This privacy policy is guaranteed by the developer himself, not a company or an organization:

As the Data Protection Officer, Richard Hughes has overall responsibility for the day-to-day implementation of this policy. The DPO is registered with the Information Commissioner’s Office (ICO) in the United Kingdom as a registered data controller.

The issue here is that in GNOME Software, users have no idea that such data is being sent or collected. An ordinary user does not expect his software center to be downloading updates from an online website without telling him so. Upon opening GNOME Software for the first time, no privacy policy is displayed and no message informs the user that such data is used and sent to fwupd.org in the first place.

There’s no indication when using GNOME Software that fwupd.org will be used to update the firmware packages.

Currently, the main developer behind GNOME Software announced that 100 million files are being downloaded from fwupd.org each month. Most of them are metadata files. This is due to the fact that both GNOME Software & fwupd are installed by default on all modern major Linux distributions, such as Ubuntu and Fedora. The developer was able to produce the following chart from the data he has:

This is a huge amount of data and accessibility available about Linux users under the hands of a single developer. The collection of such data should be opt-in, not opt-out by default in order to respect the privacy of users who don’t want to use the fwupd.org service to check for firmware updates. More importantly, at least, the privacy policy related to the collection of this data should be displayed on the first use of GNOME Software. Otherwise, users won’t know that such data is being sent.

The other issue is that up to few weeks ago, there was no way to disable fwupd integration in GNOME Software. It was just after version 3.26 (not included) that the developers added an option in the settings page to disable fwupd service. Before that, you were forced to use fwupd if you are using GNOME Software. You can’t even disable it (graphically).

According to the developer, fwupd.org is hosted on Amazon EC2. Amazon (beside many other companies as well) has donated $2000 per year to develop the project, and provides some hosting features for free as well. fwupd.org domain name is registered in the personal name of the project’s developer (if you check from who.is).

The privacy policy page mentions that the metadata users send to fwupd.org is stored up to a maximum of 5 years:

Anonymized user data (e.g. metadata requests) will be kept for a maximum of 5 years which allows us to project future service requirements and provide usage graphs to the vendor.

The developer says that the IP addresses are not stored on fwupd.org. But sadly, since this is a server-side claim, there’s no way as users to confirm such claim. Also, the machine ID collected from users’ /etc/machine-id is said to be hashed (with salt) before sharing it with vendors.

A single bottleneck for manging the hardware data of the entire Linux desktop users wouldn’t be a good idea.

A Security Concern

fwupd.org allows hardware vendors to create an account by emailing the project’s founder. Then, vendors can get information about the number of users of their hardware and what operating system & client they are using. Vendors can push .cab files to users to be downloaded later by GNOME Software. These .cab filed are claimed to be tested by a separated QA team before released to users.

The security issue is in maintaining the infrastructure for fwupd.org. If hacked, thousands of daily requests to the server and the data available could be leaked. The team behind fwupd.org claims that there are many professional teams working on securing the platform according to the industry’s best practices. However, a user can not guarantee.

We couldn’t find reports or sources on how many people in total are behind the whole project, or how many people are reviewing the pushed .cab files by vendors. The metadata stored in fwupd.org are said to be stored under a locked LUKS filesystem hosted on Amazon.

The other concerning issue is that the developers are not 100% financially covered. There has been multiple calls for donations by the developers:

At the moment the secure part of the LVFS is hosted in a dedicated Scaleway instance, so any additional donations would be spent on paying this small bill and perhaps more importantly buying some (2nd hand?) hardware to include as part of our release-time QA checks.

And another one:

The current CDN (~$100/month) is kindly sponsored by Amazon, but that won’t last forever and the donations I get for the LVFS service don’t cover the cost of using S3. Long term we are switching to a ‘dumb’ provider (currently BunnyCDN) which works out 1/10th of the cost.

Which could rise questions about the future of the ecosystem managing 100 million requests per month.

A Suggested Solution

We believe the following should be taken into consideration to solve the issues above:

  1. GNOME Software should detach fwupd as a dependency. Because if fwupd package is installed, it will auto-check for updates in the background (fwupd daemon will autostart after boot) and it will send the data to fwupd.org automatically.
  2. GNOME Software should disable the service of using fwupd.org for firmware updates by default. Users wishing to subscribe to such service should opt-in their selves.
  3. Upon activation of fwupd service, a privacy policy dialog should be displayed telling users about what’s going to be collected and why.
  4. The project’s domain name and infrastructure should be managed and ran by an organization/company. More detailed information on who’s accessing the infrastructure and working on it should be available.

The Bottom Line

Here, we do not try to question the sincerity or the authenticity of the project’s developers, but we seek to let users know what’s happening under the hood of their software center. We also believe that such infrastructure should be more transparent in the way it’s secured and managed, and how the data is stored and accessed.

In result, this ecosystem will have access to millions of Linux machines worldwide, and putting all the eggs in one basket is a concerning thing. We believe that such model was implemented with good intentions in the sole reason of serving users, for free. But privacy is becoming a very important issue. especially these days with the so-called hijacks of data around the world.

At the end, we would like to thank Richard and all others for their work on free/libre software.

Update @ 14 Apr, 03:49: The developer clarified that the hardware information of a user doesn’t get sent to fwupd.org. As such, this article was updated.

Update @ 18 Apr, 19:12: To clarify this even more, firmware matching for updates is done on client-side, not server side, meaning that GNOME Software (or fwupd) does not send hardware information to fwupd.org as this article said initially. This claim was online for around 10 hours after publishing the post before it was taken down. We would like publicly to apologize for Richard and anyone who read the article in that period for making such claim without making sure of it first.

However, it’s extremely important to also say publicly that this article (in its current form) is valid and true. GNOME Software (fwupd) was confirmed to send client user-agent, IP address, timestamp, OS distribution name and OS version to fwupd.org upon each firmware downloading process (or checking for firmware updates manually by the user). Sending 5 forms of data instead of 6 doesn’t mean that the whole thing is false. The connection to fwupd.org to provide the firmware updating service is a place of concern because it doesn’t belong to the user nor the Linux distribution’s makers (Ubuntu, Debian, Fedora, openSUSE..) and happens without users even knowing. Users knowing of this service is important, and is the main point of this article.

Just 2 days ago, and after 48 hours of publishing this post, the developers have pushed a commit to the GNOME Software code titled: “Add a warning when enabling the LVFS remote”. Here’s the commit message:

 Distributions like RHEL do not enable the LVFS by default and the legal team here say we need to add some agreement text which is shown before we enable downloading content from an external source.

These concerns has no relationship to the quality or work of the LVFS project itself. Being able to update the BIOS on a Linux system without needing Windows is a huge achievement. This article doesn’t say that fwupd (or GNOME Software) is bad, or poorly written or should not be used, but discusses “concerns” related to privacy & security, nothing more and nothing less.

Shares
Load More Related Articles

14 Comments

  1. Richard Hughes

    April 13, 2018 at 8:39 pm

    If you have a specific concern can you please email me (Richard Hughes) and we can discuss? The biggest claim here seems to be that we’re sending details of the hardware to the LVFS, but that’s simply not true; we just download a common metadata file and do all the matching client side for privacy. The service is carefully designed with privacy in mind; do you see any other web services with public GDPR statements yet? If it helps, the LVFS is in the process of moving to be hosted by the Linux Foundation, but you’ll understand this can’t happen overnight.

    Reply

    • M.Hanny Sabbagh

      April 14, 2018 at 3:38 am

      Hello Richard.

      Can you please tell us exactly if any data at all is being sent to fwupd.org? Whether it was IP addresses or client information, does GNOME Software (or fwupd specifically) send anything to fwupd.org? According to fwupd.org/privacy, in the tables at the end of the page, it says it collects:

      – Machine ID (hashed), failure string and checksum of failing file, OS distribution name and version.
      – IP address (hashed), timestamp, filename of firmware, user-agent of client.

      And in one of your blog posts, I see that you were able to produce that graph shown in the article. If there’s no data being sent to fwupd.org at all, then what’s the source of such data?

      Also, if there’s an update, GNOME Software will download the firmware from fwupd.org. But the user didn’t acknowledge this, as the article explains, there’s nothing in GNOME Software to tell the user that there’s a request to this domain to retrieve a firmware package (.cab file).

      The current point which I think you are making is that the matching is done client-side, not server-side, so my hardware-list doesn’t gets sent to fwupd.org. Is that correct? I have updated the article accordingly.

      I would be happy to update the article with any information you would like to add.

      Reply

      • Richard Hughes

        April 14, 2018 at 7:36 am

        Sure, we get the IP address and the user-agent when downloading the firmware file. The metadata is downloaded from the CDN so we see very little as there are basically no logs there. You only upload the firmware report when you’ve actually done a firmware update and you want to *opt-in* to sharing metadata with us. We show you in the console exactly what data is sent; the *exact* json string.

        I have four main problems with your article:

        * You’ve included my home address, personal telephone and personal email as part of your article. This is not necessary in any editorial context and I’m asking you to take the image down right now. I know you can get the information from WHOIS but that’s not the point.

        * It was poorly researched, and so spreads misinformation which people have themselves shared. If you had reached out before you published I could have corrected your most embarrassing errors. For instance, if someone hacked fwupd they could not install malicious BIOS files, as the capsules would no be signed with the OEM hardware signing key. They would indeed be downloaded, but would fail to be deployed. I suppose this would have produced a less sensational article.

        * fwupd.org is a site with a privacy policy, which puts it in the top 10% of websites. The web API has a GDPR report, which puts it in the top 1% (although more sites are doing GDPR reports now the new EU law comes into effect). I’m trying to be as transparent as possible and you’re using that information to spread mistrust and asking me to be more transparent (which I don’t think is actually possible). If I never wrote the privacy report would you have done enough research to publish an article on the same topic? If not, what incentive does that give other open source developers to care about privacy? Have you reached out to Canonical and asked what privacy policy they have for the NTP “ping” that’s done on every boot, on every machine? Have you reached out to the various distros for the internet connectivity check that’s done every time you connect to a new network?

        * You care very much about privacy, but privacy badger identifies 16 (!) potential trackers on this very page. Double click, Facebook, Google, and others. There is also no notice of 3rd party cookies were being collected, which is required as part of EU law. It seems disingenuous to care so much about privacy and then have this degree of data collection.

        You’ve implied mistrust and malignancy on something that I’ve spent the last 2 years building, mostly in my free time, for the Linux community. I really don’t know why I bother.

        Reply

        • M.Hanny Sabbagh

          April 14, 2018 at 8:27 am

          Good, so we both now agree that GNOME Software connects to fwupd.org and sends IP address (which is required by TCP) along machine ID, operating system, distro version and firmware requested. And we also agree that you are doing all of this without the user even knowing of such service. Which is the primary concern of this article.

          Regarding the points you made:
          1- The information is available to public. Anyone can retrieve the same data in the image using tens of other tools or services online. But as you want to remove it because it contains “private data”, I removed it from the article.

          2- I am not sure about where did I mention in the article that BIOS firmware can infect the users if website is hacked. Please read the article again. We said the requests to the firmware can be leaked and the data reserved (for a maximum of 5 years) can be leaked. Not a firmware injection or something.

          3- I am not here responsible for other developers or companies or what Canonical does. I clearly don’t work for Canonical and I don’t know what’s the connection of any of this. You are like saying: “Hey, everyone else does more than that”. I don’t care, I don’t have to investigate the whole world’s software first to publish about this. You are ingratiating a service that you didn’t tell users about and you are having some of their data sent directly to one of your websites without even a notice or a message to tell them about it. You are having the IP addresses and client user-agent requests of users of Fedora, Ubuntu, Debian, openSUSE, and all the other Linux distributions which ship your software by default. Which is why highlighting such an issue in the press instead of sending an email to your inbox is a priority.

          4. I hope you recognize that FOSS Post is an Internet website, right? Not a software center shipped by default for millions of users. Instead of telling me to look at people at Canonical or Ubuntu, or look after the trackers in my own website, you should be concerned about the issue in your software.

          Being more transparent can easily be done by displaying a message telling users what’s going to be collected (just like Firefox does) and why. It also can be done by publishing information about the infrastructure of fwupd.org, and how it’s secured and how many people are working on the QA behind it. Are they full time? Are they enough?

          I have no personal issues with you or your work. I don’t even know who you are. My only concern in this article is technical related to privacy, not to you as a person. And I cleared that at the end of the article.

          Reply

          • Richard Hughes

            April 14, 2018 at 9:54 am

            You’re still not getting it. The metadata is coming from the CDN which erases the logs after 3 days, we don’t use any of the data from the CDN in any analysis or store it any database. We only store the user agent and hashed IP address on fwupd.org when you download firmware.

            > I hope you recognize that FOSS Post is an Internet website, right

            I do, but I also know how to create a website without all those tracking cookies. I think you need to be aware that if you’re lecturing other people on a *tiny* privacy issue then you need to have your own privacy policy, and at least comply with EU law regarding 3rd party cookies.

            > by displaying a message telling users what’s going to be collected

            Do you think the average user knows what an IP address is? Or a one-way-hash? Or the difference between firmware and software?

            > Are they full time?

            I don’t see how this affects anything. The LVFS is in the process of transferring to the Linux Foundation, but that’s not for “security” that’s for other strategic purposes. For what it’s worth, I am a full time employee for Red Hat.

            > I have no personal issues with you or your work.

            Then the article must be written with a very ironic writing style. I think other people will read this article and think “why should I bother” when there’s clearly no pleasing some people even when carefully balancing all the difference UX, UI, legal and regulatory issues.

            Please be aware of the power you have writing articles like this, especially when FOSS Post claims it “supports other FOSS projects” when I can see nothing of the kind here. Sorry to be harsh, but you’ve written something that’s more akin to clickbait than a “well-written article”.

          • M.Hanny Sabbagh

            April 14, 2018 at 3:02 pm

            This is exactly the problem, what you just said:

            > we don’t use any of the data from the CDN in any analysis or store it any database. We only store the user agent and hashed IP address on fwupd.org when you download firmware.

            Your ability of having such data in the first place without users knowledge is the problem. Your making of a connection between my personal computer and your own website to download a firmware which is reviewed by your own QA team without even telling me all of this is the problem. That is the issue which I hope you can understand from my point of view.

            I am not going to reply on anything regarding the website as it’s off-topic. The website’s complying with EU laws and 3rd-party cookies is not affiliated in any way to the content which is being published. The availability of the privacy page doesn’t mean this is true or false.

            > Do you think the average user knows what an IP address is? Or a one-way-hash? Or the difference between firmware and software?

            Your reply here is shocking. You are like saying: “The average user doesn’t know what an IP address is, or what a firmware is, so why bother with telling him that such connection and service exists?”. If a user doesn’t know what an IP address is, or what a cookie is, or what a user-agent is, or what anonymous hardware information is, this doesn’t mean we can do anything we want with the data, just because the ordinary user doesn’t know.

            If every software on my Linux distribution will do the same, it would be a major problem. You would tell me that Ubuntu is doing it with NTP, but that’s not our focus here right now. I’ll try to publish about these topics later.

            I believe many privacy-caring people who are also using Linux on daily basis would also be concerned as me. I didn’t call your program a “spyware” and I didn’t say you collect personal data or for marketing stuff. All what I did was reviewing the issue using sources you’ve already posted yourself. The audience is free to follow the issue their selves and decide whether it’s ok for them or not. As a journalist, and as a Linux user, and as a free software developer as well, my job is to let people know about the topic, then, they can do whatever they want.

            Thank you for your feedbacks and your work. I appreciate it despite such concerns.

          • Matthew Garrett

            April 14, 2018 at 10:02 pm

            > – I am not sure about where did I mention in the article that BIOS firmware can infect the users if website is hacked. Please read the article again. We said the requests to the firmware can be leaked and the data reserved (for a maximum of 5 years) can be leaked. Not a firmware injection or something.

            The article originally included this:

            “If hacked, millions of Linux machines can be vulnerable to firmware malwares which can cause permanent hardware losses”

            You deleted that, but Richard was clearly responding to the original version. It’s amazingly dishonest to pretend that you didn’t make that claim.

          • M.Hanny Sabbagh

            April 15, 2018 at 3:21 am

            Nope, the article was already updated when Richard made the comment. This discussion started around 9-12 hours later of publishing the post, where the older version wasn’t already there in less than an hour of publishing the post.

          • Matthew Garrett

            April 15, 2018 at 8:46 am

            The fact that you made that claim was repeated elsewhere, and Richard was responding to that. But I guess you’re admitting to having said the thing that you denied saying?

          • M.Hanny Sabbagh

            April 16, 2018 at 10:14 am

            If something was removed from an article after 30 minutes and someone comes after 12 hours to respond to a part which no longer exists already, then there’s no point in discussing that part. A reader for the comments would assume that such part still exists at time of discussion, which is not true.

  2. Mark Braught

    April 14, 2018 at 1:17 pm

    Richard Thank you for all of your Work!!! After reading this full article and all of the comments this guy is seemingly attacking your work for no apparent real reason.. if he worried about privacy there is a whole lot of other places to look before attacking you

    Reply

  3. anonymous

    April 16, 2018 at 7:09 pm

    This article is clearly not well researched and only serves to damage the reputation of fwupd; a great tool that easily enables firmware updates. Richard Hughes has done a lot for the community and just recently I used his work to update a controller I bought for my SNES classic:
    https://blogs.gnome.org/hughsie/2016/08/18/updating-firmware-on-8bitdo-game-controllers/

    If you’re really an open source software enthusiast I hope you’ll retract this article and discuss any issues you have with Richard directly instead of needlessly damaging the reputation of an open source project like you have.

    Reply

  4. Richard Hughes

    May 3, 2018 at 8:16 am

    > Just 2 days ago, and after 48 hours of publishing this post, the developers have pushed a commit to the GNOME Software

    Dude, you’re making the same mistake *again*. We started working on this feature about a month ago, well before you posted this article. See https://github.com/hughsie/lvfs-website/pull/102 for the discussion. There are even older PRs for lvfs-website before that one too.

    It was being driven by a requirement inside Red Hat, which disables the LVFS by default in RHEL by default. The requirement was to make it easier to actually enable and also explain to the user that it was a 3rd party service not provided by the software provider.

    I think you need to stop trying to dig yourself out of a hole. You think you’re making yourself look like the champion when in fact it’s just delusion. Sorry to be blunt.

    Reply

    • M.Hanny Sabbagh

      May 6, 2018 at 7:17 am

      There’s no mistake at all. The commit was indeed submitted just two days after publishing the post, whether such thing was affected by the post or something that happened earlier is open to debate and doesn’t actually matter, but the fact remains that it was indeed pushed to Git just 48 hours after this post.

      I hope you can keep your insults in your pocket, we don’t need them here.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *