Should we add `.hex` to the upload whitelist?


(Pharap) #1

We’ve had a small handful of people want/need to upload .hex files lately who have resorted to the "pretend it’s a .pdf" trick.
Should we perhaps just whitelist .hex (and possibly .ino) files to avoid this?

On the one hand I don’t want to encourage posting just the .hex and no source code,
because I think the availability of source code is very important for the Arduboy,
but I think it would ultimately make life a lot easier.


And here’s an anonymous poll for people who want to chip in without having to comment:

  • Yes
  • No
  • Maybe

0 voters


#2

I think it would definitely be good to at least add ino to the whitelist just in case anyone doesn’t want to identify github or some external platform to post their games to here.


(Kevin) #3

Strong in the maybe category mainly because I am trying to think of what the negative consequences are? That people just start posting hexes? I mean that seems to be happening anyways just abstracted. You know what would be cool is if I could code a special popup window that was like “Think about the community before posting closed source” but I don’t know how to dev that hard on discourse.

Probably can add.


(Pharap) #4

I’m mainly thinking of using .inos only for really small demos.
Any decent sized game should span several files,
and anything more than about 3 files becomes a pain to host on the forum.

If .ino, .h and .cpp were all allowed then it would be a lot easier for people to post larger code snippets (versus uploading them to GitHub as gists),
but at the same time you don’t really want to encourage people dumping too many code files on the forum because the forum isn’t built for navigating code files (whereas GitHub is).

It would be more viable with a way to limit the number of code uploads in a single comment and/or to limit the number of lines in the code.
(Number of lines is a terrible metric, but measuring actual complexity would require parsing, and C++ is notoriously hard to parse.)

One big plus would mean a bigger delay between “learning to code basic stuff” and “this is GitHub”.
GitHub can be scary and overwhelming for beginners, so allowing them a bit of time to play with code and not worry about hosting would probably be a good thing.

As far as I can tell the only negative consequences are:

  • Potential for people to upload bad .hex files
    • Though this is already possible, nobody vetoes what users upload to their GitHub pages
  • Potential for .hex files to contain viruses
    • This could happen with any file though… (.pdfs are possibly an even bigger risk than .hex files)
    • The .hex format could be verified with a line-by-line scan with a regex
  • Making it easier for people to release closed source games
    • They could do this anyway with pastebin, GitHub or gists
    • Unless the forum adopts a strong “open source only” stance, this has to be allowed (albeit still discouraged)
  • Storage space
    • Hex files are usually pretty tiny anyway, and easily compressed
  • Multiple hexes being offered, people downloading the wrong ones
    • The only problem that can’t be mitigated beyond good signposting
    • People posting any more than 2 hexes (devkit and regular) should be using GitHub or GitLab anyway

Positive consequences:

  • No more .hex.pdf files
    • These are confusing to non-technically inclined users
  • No more links to dodgy file sharing sites (for .hex files at least)
    • This actually is a genuine risk vector for forum users
  • No more "how do I upload a .hex" questions from beginners
    • We still have to explain about GitHub and GitLab for code… we could do with having an article about that somewhere (as well as one for licences)

I’d help if I could, but if I were to help I’d probably end up just trying to simultaneously consult the docs and relearn Javascript,
which is something anyone could do.

(Also I’d be reluctant to ask on the actual discourse forums because last time I visited I was less than impressed with Jeff Atwood’s conduct.)


(Matt) #5

Once a hex is uploaded here, is it possible to get the url to it to plug into ProjectABE?


#6

I voted no. Because I think it promotes closed source and posting a hex file will not be useful for troubleshooting.

I’m in favor to support .zip attachments so complete projects can be attached.


(Simon) #7

This is why I voted no. A single thread could contain many versions of the same .hex file as it is developed and released. Some of the ‘work in progress’ hex files may be problematic or downright dangerous.

I don’t mind .ino files if the user is trying to demonstrate an issue. However for beginners the code is typically small enough to post inline and for advanced users they are probably using GitHub already. So no for .ino as well.


(Jean Charles Lebeau) #8

The problem of the zip is that they can be infected or have some inapropriate content.

I have voted yes as even if the hexes aren’t the main important, some could only want to share them and it’s their choice. I’m alot more for sharing the sources. It’s the spirit of this console and i like it but what is better: not sharing the programms here if you don’t want share sources ? Not sure that it’s better.

Sharing the ino is fine too as it’s can be easier to share a prog or a part (a .cpp or a .h too) than sharing a tone of text when the code is optimized or for correct an error. Even for sharing the source, it’s can be referenced by an URL or given in a forum if the creator don’t want use github.

For the zip it’s better that the zip with .hex, sources and documentation is done by a known person. It’s can be imagined than bots or bad intended personn could send infected zip.

Else atm i think that the restrction is more a brake than a good thing but i think too than authorise these files will not change the forum in an important part. The main users will continue to post their sources like they do and give link on them and the .hex


(Kevin) #9

I will look into it, I want to see if I can implement some kind of warning or notification when the user wants to put up a hex, but not allowing it is kind of silly because people still can share a link or distribute it otherwise it just kind of wastes time. Theoretically the best implementation would be if you upload a hex to bring in an embedded version of the projectABE emulator. Which, is kind of what I’ve always wanted to do ala the pico8 forums.


(Pharap) #10

It should be, in the same way it’s possible to get image urls.


There’s nothing to actually stop people publishing closed source games as things are.
There are actually a handful of Arduboy games that are closed source (or ‘source available’ rather than open source).
Most are games that have bypassed the forums and just been published on Twitter, but there’s also that recent Starfox-like game (I asked about source code at least twice and got no reply) and Circuit Dude (‘source available’).

Ultimately all we can do is encourage people to publish and licence their source code and (if you’re hardcore) refuse to play games that are closed source.

I think we’ve got enough of a culture of source-sharing that we don’t have to worry too much.
Pretty much all of our biggest contributors publish their source code, so most people will follow the example.

No, but if someone wants help with their code we will just tell them “we can’t help if we don’t know what the code looks like”.
When they don’t get help, they’ll either publish their source code or stop asking for help.

This is also why I was considering source file uploads.

As @Jean-Charles_Lebeau said, .zip files are a lot easier to shove viruses into.

A .zip can contain literally anything, which sort of defeats the point of a whitelist in the first place.


Practically anything that could happen from a .hex uploaded to the forum could already happen from a .hex uploaded to GitHub or Pastebin.

If a .hex is found to be problematic then the same thing would happen as has happened in the past (anyone else remember the fidget spinner incident?) the offending .hex would be removed and an announcement would be made informing people of what has happened and offering assistance for anyone affected.

Hopefully something like that could never happen for newer Arduboys because they should all have their protection fuses set (though I wouldn’t be surprised if the odd batch sneaks through).


#11

My answer was in the context of allowing hex files and not .ino

pdf files can be mallicious as well. But new users are not able to add attachments afaik so we should be save from those kind of spam bots. Mallicious users could stil upload mallicious stuff though. This makes me wonder if dicourse has any virus/malware scans. If there is no protection. Maybe pdf should be blocked too?


(Pharap) #12

So you’d allow both .hex and .ino (and/or .h and .cpp), but not just .hex on its own?

This has crossed my mind a few times.
I don’t think many people have ever actually uploaded proper .pdf files.
(The only one I can think might have a .pdf linked is Dark & Under, either from its processing prototype stage or the .pdf produced when the game was finalised.)

Fortunately I think most .pdf viewers disallow the more dangerous features by default,
but there will always be a chance to exploit bugs.

That’s a very good point.

There have actually been one or two cases of companies trying to advertise things on the forums,
but regular users usually never know about them because the system is usually clever enough to prevent suspicious threads from being posted and flag them up for moderator approval.

There’s also a few false positives from time to time,
which is why there’s the option for moderators to accept or reject the post.

If we ever get a user malicious enough to willingly post a virus then we’ll defintitely be banning them and most likely will be passing their IP address on to the relevant authorities because that’s illegal.


I just had a look through Erwin’s Game Repo,
and it’s actually surprising just how many games are marked ‘proprietary’.
In a lot of cases the code is available but just doesn’t have a licence,
but there’s still a few that don’t even have the source published.


(Scott) #13

Does the forum allow .arduboy files? (I know it allows .zip but will it allow them with the .arduboy extension?) If not, and we decide to allow .hex then I think .arduboy should be allowed too.


(Pharap) #14

I don’t think it does allow .arduboy files.

I suspect we’re probably still on the default settings because I know of other Discourse channels where uploading .zip files is possible.

.arduboy would make sense to add, though it does suffer from the problem of being a renamed .zip file,
but equally I think that’s going to be less of an issue because it’s less likely to be treated as a .zip by default,
so if anyone runs into an issue it’s more likely to be someone with enough technical knowledge to realise “that’s an executable, I should probably be careful about that”.

Not to say it’s a non-issue, but it’s probably marginally less of a threat than a plain .zip.


(Kevin) #15

I’ve been meaning to poke around and see how feasible it is to make the forum do something special for these types of files but haven’t had a moment to sit down and fiddle with the forums.


(Matthew Vicaradge) #16

As a recent new user I posted a .hex.pdf file because I didn’t know any better, you already have a way of posting code snippets in posts for any questions. Perhaps a starter guide for new users to read and understand the etiquette would reduce a lot of the .hex.pdf stuff?
You could create a thread with it in that unlocks posting ability after it’s been read.
As a noob I would have found it useful, I guess as seasoned users you end up explaining the same starter stuff questions over and over. (Which I’m super grateful for by the way).


(Kevin) #17

Yes this is a good suggestion!

Edit: Well they turned the search and hamburger icons to gray instead of white so now I have another reason to do some development on the community site… standby…