related threads

http://www.hedgewars.org/node/3051

http://www.hedgewars.org/node/2238

http://www.hedgewars.org/node/772

http://www.hedgewars.org/node/615

The last three threads are only about maps, and I only indirectly talked about maps (as forts) so those are only kinda related.

Only the first thread is really related, it is over 3 years old and it quickly derailed and the original poster has also given a different approach.

Soooo …
Any specific comments about my suggestion with my approach?

In Trackmania, there's a locator file system. Basically, instead of uploading custom content, you provide a locator file with a link to download content from some website.
Let's say you have a hat named MyHat.png. You upload it to e.g. Imgur, and DIRECT link to image is imgur.com/MyHat.png. Then in the directory with your image, you make a file: MyHat.png.loc. The file contains only one line: imgur.com/MyHat.png, the link to your image. In multiplayer, another user encounters your custom hat, but you provide a locator file, so engine will download the file from you and then download hat from provided link.
Well, not that it's some very good approach or something, but still may be considered, as it at least minimizes amount of uploading done from custom-contented client.

I don’t think this is neccessary at all for flags, hats and graves, since those files are very small.
And I am not very happy with that approach, I don’t think it can be reliable. What if the file is gone on the server? I don’t like the idea of summoning a third party either, because you just added another single point of failure to the system.

Agreed.
I predict that's the way it will happen, eventually, one day:

• Users upload their content on hedgewars.org, specifying type for each upload (flag/hat/map/theme/etc). Probably it will also be possibe to upload multiple content of same type as .zip/etc (in case you want to upload more than one hat at a time).
• Uploaded content goes into a moderation queue, which is checked by developers and probably also few trusted individuals.
• Content that makes it through the queue without getting rejected (inappropriate or otherwise unfitting content) will be automagically downloaded when-needed to clients who opted-in to doing that. (Possible two opt-ins for "game-style" and "anything really", probably can also filter which types are downloaded, or as Wuzzy suggested - what max size).

@sheepluva: LOL, you are funny. We already have a very similar system. The only differences are that its not on hedgewars.org but on hh.unit22.org and bender is the one who accepts and rejects stuff.
There’s even kinda an integration to Hedgewars. You find a link to a dumbed-down hh.unit22.org page in the DLC section, from there you can download all published HWP files.

The only thing which would really change with your “plan” is that players don’t have to go into DLC beforehand, instead their client downloads it when required.

But the thing is, that’s not what I have suggested, sheepluva. I rather have the clients download it from the other clients, without any middlemen in-between.
If a player consented into downloading user-created stuff, that player should be prepared to see stuff they may not like. It’s the Internet, deal with it! :P

For example, it should be possible to share a simple flag without all that bureaucracy IMO, middlemen just slow everything down and, frankly, there is always the danger of censorship.

It's not aplan, it's a guess of how things will be in future.
And yes, the biggest change to the user would be it being done automatically - which is a very significant change to the user experience and significant convenience IMO.

Hm. Flags are extremely tiny so "direct" transfer would be the most possible there. Could probably embed flag image in team info.

"direct" in quotes for an important reason: there are no direct connections between clients (for technical reasons as well as security reasons) - don't forget that all traffic goes over the game server!

So "direct" transfers (instead of some http download in the background) would be an additional burden for the game server, but yes, it would be more convenient to the user if their flag would be automagically uploaded/transfered from the client.

Censorship is not a concern IMO, since Hedgewars is not and should not be a place for controversion political, religious, etc. exchanges to begin with.

I consider possible censorship in this case a bonus (don't quote that out of context...) that allows us to avoid part of the community to go into full retard mode on official server and e.g. put genitals, swasticas and other stuff all over the place (because that's very funny and original apparently).
Also a moderation system allows us to throw malicious stuff (corrupt images, that contain viruses) out of the queue before any client can download it.

At present, until we add a lua sandboxing system, the consequences would not be simply downloading copyrighted content, or an illegal or pornographic image. It would be fairly simple to do very bad things to your computer.

Once we have sandboxing, might be ok with an opt-in advanced config setting with clear warnings. At present, very much opposed.

Current setup is not that onerous, and if you can't be bothered to go through the trouble of proper packaging... Well, just saying, it improves the SNR.

Either Hedgewars is apolitical, then you don’t care about political stances at all. Is this is your position, then you don’t have a opinion about censorship to begin with.
But you have a clear position on censorship and therefore your claim that Hedgewars is completely apolitical is wrong.
Even worse, by claiming that political discussions are a taboo on Hedgewars while in the same moment you engage in a political discussion you have applied a double standard here.

Oh, yeah! Since genitals are so dangerous! Come on, what’s the big deal with that? You have a problem with that because society has conditioned you to do that. But as soon as you start to think about it rationally, there is no real reason to have a problem with that.
Whatever. If you seriously have a problem with that, okay. In my suggestion I have explicitly adressed such concerns: All users have to opt-in! So all you would have to do is to simply not opt in. It’s all about choice. Therefore, there is no good reason to take away choice from users.

Therefore, you have absolutely no high moral ground to justify why you or anyone else should be privileged to decide what goes and what not.

So much for Hedgewars being apolitical. Anti-nazism is a political position.
Do you know what’s funny? If your goal is honestly to keep Hedgewars apolitical and areligious, you have failed miserably.

Hedgewars has a flag with the “anarchy-A” on it, a symbol of anarchism: cm_anarchy.png.
cm_soviet.png shows the symbol of communism.
So Hedgewars is OK with including a symbol of communism AND anarchism, but nazi symbols are stricly prohibited. Yeah, that makes sense!

Hedgewars contains several crosses as graves: Bone.png, eyecross.png, Simple.png (the name sounds like the cross is somehow the “normal” grave stone). coffin.png has a small cross painted on it. The cross is of course a symbol for Christianity. v0.9.20 ships 32 graves, 4 of them are or have a Christian cross. That’s 12.5%!
Logically, I have seen a lot of teams using a cross as a gravestone. Even a couple of my teams use a cross, and hell no, I am not a Christian.

And nobody, literally NOBODY I met online had ever had a problem seeing that symbol plastered all over the landscape and I am pretty sure Hedgewars is not only played by Christians.

There are even two themes directly linked to Christianity: Christmas and Halloween. (OK, the latter only kinda linked to Christianity, but still!)
Also, there are two easter eggs in Hedgewars which change the background (among other small modifications) of the main menu on … wait for it … Halloween and Christmas! (I haven’t found other easter eggs yet).
The flag cm_eyeofhorus.png shows the Eye of Horus and cm_pentagram.png shows a satanic pentagram.

Hedgewars includes christian, satanic, anarchist AND communist symbols by default, which is kinda funny when you think about it.

Lastly Hedgewars has a large collection of national flags, and they really didn’t get into this game by accident. And any claim about Hedgewars somehow being apolitical crumbles.

All this was no problem for you developers to include into Hedgewars.

I am listing all this here not to condemn that Hedgewars included all those symbols and references. Not at all! Just please think about it why the inclusion of those symbols was not a problem.
Guess why? Because of context.

Just because your team has a cross as a gravestone doesn’t mean you embrace Christianity. Just because your team uses cm_soviet.png as flag doesn’t make you a communist. By extension, just because your team has a swastika as flag doesn’t make you a nazi.

You cannot have Hedgewars have a couple of political and religious symbols and then somehow selectively reject some symbols because they are belong to some ideology.

To have any moral high ground here, you would have to wipe out political and religious symbols out of the official Hedgewars builds, which would mean the loss of 2 themes as well.

Of course I would not want that at all.

More precisely: Other stuff you personally don’t like.

First: “copyrighted content”, “illegal or pornographic image”.
This is a pretty arbitrary notion. The problem: What material is copyrighted depends on the jurisdiction, same for “illegal” images and pornography (and the legality of pornography). Even the definition of “copyright” itself varies per jurisdiction.
So do you seriously want to play the copyright police? And if yes, which state law do you pick? USA? Germany? France? The country you live in? Whatever the choice, that choice is likely to be completely arbitrary. Same for “illegal” images.
Pornography is special here, since you didn’t make an appeal to law here. You just implied that pornography is somehow in itself bad, without any justification. Just like sheepluva you suffer from a cultural bias which conditioned you into condemning pornography as such.

Second:
What does all that have to do with Lua?
I am not even talking about sharing scripts online (where I could understand the concerns). I am only talking about the small graphics, nothing more.
I don’t understand why this would put my computer at risk.

You realize you're overthinking this?

Also a lot of accusations there.

• I specifically mentioned why i consider moderation of things like genitals depictions a good thing. (Also there are child protection laws/concerns out there). But instead of responding to the reason I mentioned you attack me personally.
• I never claimed that my "goal is honestly to keep Hedgewars apolitical and areligious". I just stated my opinion that it shouldn't be a place for such. Maybe you took the "shouldn't" more serious/restrictive than I meant it - I'm sorry for the misunderstanding in that case.
• I'm not against swasticas in Hedgewars because of political reasons, but because of trolling and unnecessary drama that it will cause.
• I never claimed to be on higher moral ground, I just stated my opinion as such.

The official server is a place for players to find other players to play with.
Most people are there to have fun playing the game.
If somebody wants to discuss controversial topics even if it might offend other people - then he has to expect to be the loser of any argument/fight:
There are many many many places where he can do that without ruining the fun for players in the official server.

Also most of us developers probably have better things to do than permanently comforting offended players and handling reports or even legal issues.

PS: You didn't comment on malware distribution...

PPS: I find your arguments about copyright and pornography a bit irrational. You seem just to argue to argue :P

Sure laws vary from country to country and state to state, but you seem to ignore that there are also international laws and also extremely common/similar laws and morals in most parts of the world, especially when it comes to explicit material in any combination with minors.

Also we should at least try to stay within the laws of the server's and developers' locations.

It's easy for you to argue you want the freedom for players to upload anything, but at the same time you seem to be completely forgetting that we developers might also want the freedom of not being in jail.

Just because we can't make everyone on this planet happy at the same time, doesn't mean we can just go forward and ignore everybody's laws, opinions, morals and feelings.

Well i think i need to post my oppinion on this...

As it is now when hosting a room with custom-made stuff, its like:

Situation 1. [The content is in the DLC] (me) "Hi, sorry you need to get the content from the DLC." Then they usually respond with: "I got the content!", and I know that he doesnt. As i know now they probably think its the game i mean. They probably updated from 0.9.xx to 0.9.latest and it was a huge achievement for them. Then i respond with "Well if you want to play this, then you need to blah ... blah ... blah ...", and then the player leaves.

Situation 2. [The content is elsewhere] "Hi, if you want to play this, then you need to download it from http://scary-looking-website-for-them/content.hwp and place it in ... ", then the player leaves.

Well i know that alot of players know how to get the content from the dlc or elsewhere, but just now I want to focus on the ones that dont.

I think that the best solution for this is:

A. A popup that says: Hi there, you are missing some content which is used in this lobby/game. Should I get it for you? Then the player can select between Yes and No. If No, then the player should sit in the room and wont be able to add teams, and it will show some kind of red text saying that he wont be able to play.

B. Same as A, but only for DLC content.

C. Same as A, but will display if the content is "trusted", which means that its in the DLC or not.

D. Skip the popup idea but use similar solution
======
I dont think that some sort of "in the backround auto-download" is a good idea. Neither do i think that only the DLC is good either. However the DLC is useful for the players which usually isnt connected to the web.

Two years later, I still think this is relevant. I was playing online recently and I thought it would be fun if I could use my themes in my room, so people could join and play with them. I also had a case where I joined with my brother in a room with Luelle theme pack. I have it, but he didn't so we couldn't play together there. So it would be nice if themes could be somehow transferred to clients.

Now, there are many concerns regarding that. The least important one is probably server traffic. This can be completely avoided by using the locator system I mentioned few posts above.
\/\/\/\/\/\/\/
I will now assume the use of locator system in the rest of this post
/\/\/\/\/\/\/\
The locators can even point to specific .hwp files now (perfectly, each .hwp could contain a locator, so they would be automatically downloaded when missing).
HOWEVER, there's a problem with this too :/ Because .hwps can contain more than one theme. E.g. if someone used Luelle (or my) pack, they'd need to download all themes to play on one. So, that's a problem.
Another one would be big themes. Since music or backgrounds are usually the biggest part of the theme, ideally only the most important textures (land tex, girder, border, objects) should be downloaded first and less important ones could come during match or be disabled from downloading. But it's impossible if files sit inside a pack. Well, locator files could be per-theme, and each theme could point to a package. Or even two, with one of them being the "mandatory" one, important gameplay-wise. Authors of themes should then provide the downloads themselves. Maybe it wouldn't be that bad. DLC server hosting for locators is neat idea too, as DLCs are there anyways.

Second concern is security. The most important thing to provide regarding this are security options. They would be:
-disable downloads (so you need to install everything manually)
-allow only official DLCs
-allow downloads from "trusted" users
-allow all downloads

Few clarifications here. Official DLCs mean locator files that point to files located on official DLC page, or any trusted website, like hh.unit22. "Trusted" users mean users registered on the main site. Well, there might be strangers there, but if you spend more time online, you actually get to know who can you trust (like you have some friends from the forum etc.). And of course, there should be an option to report users for malicious content, so users could have their account banned from sharing, thus removing them from "trusted" list.
Alternatively, "trusted" users could be registered users chosen by admins. Looking at the forum, there would be, like, 3-4 of them. Helpful option if someone doesn't have his themes on DLC page.

Now, this ancient quote:

^This
Totally. Client should scan room files if he's not missing anything important. And then, there should be warning shown both to client and room owner. Currently, if some client is missing files, they will just disconnect on start. If room owner is aware of missing files, he can choose a different theme/map. (well, there's also a case when they can have different theme/map versions, but that only could be resolved with some internal versioning or automatic checksum)

One more thing about the transferring, since it would likely consist of packages, they obviously can just be stored in package directory. But maybe if someone doesn't want to clutter it, they could go to some directory called "cache", which would be purposed for storing auto-downloads.

Now, after this lengthy post, what is the current stance on file transferring? Is it still some planned feature, or some foggy feature, cuz no one know how would that work? (I think the system I just proposed is fairly robust)
Or is it just too much work? I'd be glad to help, but I have no slightest idea where to even start

That. Ain't nobody got time for that

The how isn't that much of a problem actually. It can be figured out and there are many other games one could take as example for how certain concerns are addressed, additional to the already great ideas in this thread/forum.

Let's say I'd like to at least try making this. I assume it will be mostly frontend thing. I didn't even touch the frontend yet, so some reference would be nice. AFAIK, engine isn't documented at all (unless there's a page I'm missing). All contributions so far I did using grep and CTRL + F. So I'd like to know which files should I be interested in the most.
I need to know which method/file is responsible for loading .hwp files. Also, if there's already some function that can be used for file transferring. Also, file(s) containing and setting interactions with in-room view would be useful too. I'm looking at it now, and I don't know what is what ;_; (engine alone was far more intuitive)

EDIT:
k, I found that pagenetgame.cpp is the room page. I also located the widgets etc. I have to figure out how to use theme/map and check if the client has it.

EDIT2:
Ok, I can now detect if client has the theme used in room. Now I need to handle the server signal of changing theme and re-detect it then. I'll move to maps, or even hats, later. Same with warnings. First I want to figure out the transfer to see how hard would that be. I also realized you can't load new .hwp files while the game is running, which will be a problem too ;_;

EDIT3:
So, apparently there's a download option on DLC page. Seems like I'll be able to use this code.
So here's my plan for handling this:
-Client gets message from server about current theme
-He checks if he has such theme
-He doesn't
-Message is sent to server that theme is missing
-Server receives the message, room owner gets some warning flag to know that a player has problem
-Now, server goes to theme directory and looks for locator file
-If it finds one, it sends its contents with a message, otherwise an empty locator message is sent
-Client gets the message
-If he received locator, light bulb icon changes to download button
-Pressing the button starts fetching the file from provided locator
-When file is downloaded, the pack is loaded and client sends message that he is ready
-Everyone is happy

Installing the pack seems the most difficult, because currently the game needs to restart when this happens. I'd have to fix that and it's more than copy-pasting the code ¬_¬

EDIT4:
Is there any particular reason why server is written in Haskell?

EDIT5:
Also, how to rebuild server? make install "compiles" the files, but nothing actually changes. Why is that?

Nice to see this discussion now being about actual implementation details rather than fear about that someone might be offended.

I dislike the model of “trusted” users, it kinda creates a two-class system. Someone has to approve all the HWPs manually, and people around Hedgewars are already busy. So I guess it won't happen or will be a very inefficient system and it would be hard and time-consuming for Lua coders to get anything approved.
Also, it can't prevent man-in-the-middle attacks or social engineering.

Also, I think your suggestion may be too extreme. Certainly there are HWPs which should be of no concern, like a HWP with 1 flag only. However, there might be still reasons the user does not want automatic downloads (like saving disk space), so adding it as an option might still be OK.
The real security concern is about HWPs which contain Lua code.

So I have a better idea: Check the content. If it only contains simple images (flags, hats, graves, etc.), it's OK by default since it does not contain executable code. HWPs which contain Lua code should be dealt with much more strictly, of course.

What is most imporatnt in MY opinion is “hardening” Hedgewars clients against any kind of “evil” HWP. Sure, we could try to avoid “evil” HWPs being distributed but since we're in the Interneet, there will always be channels to get HWPs somewhere else. So only checking the sources is not helping.
The easiest way to get an “evil” HWP on someone's computer would be through social engineering (“Here, my friend! Install this totally trustworthy HWP which will totally not wipe out your harddrive.”).
The idea of hardening is making sure the client never will do stupid things (like wiping out entire directories), so that “evil” HWPs simply can't do any damage.

Hardening the client could happen by disabling certain Lua function calls, making it impossible to overwrite particular special/important files, and so on. In short, the scope of HWPs must be cleary defined and Hedgewars must simply be unable to do anything more than that. For Lua code, only a limited set of functions must be available, dangerous functions like file.write are forbidden.
This alone would deal with a lot of threats. Lua code might still be dangerous because we would have to trust the Lua interpreter to not have any holes for exploitation. But it would be a very good start.

Note: I am not a core Hedgewars developer, so sorry if I missed some details about HWP implementation or if I overlooked a security threat. These are just my humble opinions as a humble user.

I am not sure, but I think yes. It's probably because the developer was most comfortable in writing Haskell code at the time. At least I know the engine is written in Pascal because unC0Rr was familiar in Pascal when he created Hedgewars.
I think this was a perfecty reasonable decision. And I think Haskell and Pascal are both reasonable and usable programming languages. I rarely object to programming language choices and just deal with it and suggest you to do the same. Unless it's PHP, PHP is awful!

So, if I were to add some completely new module, it would be written in Ruby
The biggest problem with multiple languages is the amount of dependencies you need to compile the code. I have 2 HW repos on my VM and all necessary libraries, and somehow it all uses 7 GB (including OS)

And the problem I see with Haskell is that it's functional language. I can stand Pascal, I really don't like this language, but it's understandable at least. Haskell however looks nowhere like any of programming languages I learned so far (and there's like 10 or more of them).
Luckily (hehe) I can't rebuild server for some reason, so I don't need to touch its code (I'm hacking chat to send my own commands).

btw, right now I'm focusing on themes and maps, because they are the easiest to send. You can just put a file into theme/map directory telling where to download it from. I'm not sure about flags and hats. Hopefully, there's a way to check which HWP they come from, because they mostly come in groups.

EDIT:
Any ideas how to get the downloaded file name? When I download from Dropbox, it redirects me to new link with the file. I made the game handle the redirects, but the new url doesn't contain file name. Anyone?

EDIT2:
Ok, so files without extension are assumed to have non-unique name, so they will be renamed to "CachedData.hwp". Most players will care about HWP contents anyways, not name.

unc0rr said he picked haskell for security reasons I think. And just liking the language I suppose.
He's open to a rewrite in rust I think ☺

I don't know how does this work.
There's this message: "At present, Hedgewars must be restarted to apply new changes and load the new content from your config directory."

When you download file, you get something like: "[PHYSFS] Mounting No such file or directory". Understandable, you need to restart game to make it work, right?

Nope.

When you download theme, it doesn't appear on the list.
When you download map, you get the same PHYSFS message, but it WILL appear on your map list. Unless you used the list before downloading.

So, quick test shown that when you download and save the HWP, even if mounting "fails", you can then play the game normally (with themes too). Which means that I was successful in setting up the transfer and you can now auto-download missing themes/maps to play together

Now I need to make a proper UI. Right now there's only progress bar.

Keep in mind that Frontend and engine are two different things.
When a game is started, the engine process will be started and it will have it's own physfs and do it's own mounting, possibly mounting .hwp files that are not mounted into the frontend yet.

That would make sense, but it doesn't explain why I can see newly downloaded map on the list.
Also, I just tried resetting ThemeModel, and after loading themes again, the new one appeared on list.

So it seems like that notice is obsolete, as HWPs DO load on download, but the game just doesn't reload the lists. Setting m_themesLoaded to false and loading themes list again does the trick. I didn't touch maps and other resources yet, but it's possible that they are ok too.

btw, PHYSFS seems to report "no such file" only on first mount. That's interesting too.

EDIT:
k, seems to be done. I'll post the preview when I merge it with my git branch. It won't be final version of course.

INTRODUCING: THE LOCATORS!

Let's say you play online and join a room with a custom theme, that you don't have.

You can't start your round, because the game will crash. So sad

But in fact, you can turn on the light bulb, like you were ready. But you aren't, that's confusing. So your light bulb disappears.

Uh, so how does it help exactly? Well, it doesn't. You just go to un-ready state and can't set it to ready.

BUT, your client is smart and sends a request for the missing theme. Room admin takes the request and looks for something that would help you. Oh, there it is. A locator.

So, what is it? Let's look inside.

It's just a single link, pointing to where should you download the theme from. Let's send it to our poor client. (everything is done automatically behind the scenes)

The locator arrived, your button just gains a new icon.

Let's click it.

A mysterious progress bar appears and then disappears. What just happened? There's a new file in your data directory.

Yea, some websites are bad at providing the file name, so a new one needs to be generated.

Anyways, that's your theme. Your light bulb is now back.

You can turn it on, this time for real, as you can enjoy playing with the new theme. The icon doesn't update at first, but when admin changes to a different theme and back, it updates correctly.

THANKS FOR WATCHING!

This works both with themes and maps. When there's a proper theme at first and admin changes to missing theme, your client will automatically be put to un-ready.
Of course when there's no locator provided, the ready button stays red until the missing theme/map is changed.

Also, if you look at Dropbox link, that's not the correct link for direct download. But, I added a special handler for Dropbox, so links are changed to ?dl=1. If there are other services that need this, they can be added too. Also there's support for download redirecting, as the provided Dropbox link isn't the final one, and also that's why there's this stupid filename problem.

See here for the code.

TODO:
-handling other resource types
-security options I mentioned before
-warning icon for room admin to know that someone is missing resource
-proper text description for new icons

_

EDIT: I have an idea how to handle minor resources.
Of course, perfectly they could be just transferred between clients. It's doable, probably using some byte stream. But... that's complicated. Also, why not be consistent?

Flags, hats and graves aren't stored in separate directories. That's a problem if I were to use the same system as with themes/maps. HOWEVER, my idea is just to store the locators in flags/hats/graves directory, but with different names. These files would start with locator, and by convention they could be named "locator_". These files would contain the url, but also a list of all flags/hats/graves from the HWP. This information would be stored in a dictionary (QHash), so if any resource from the list is missing, whole pack would be downloaded. They are small anyways.

There would be slight difference here. If you miss some flags/hats/graves and download is not provided, you can still set to ready state, as they are not needed to play the round.
The same system will probably be used for forts and scripts, and voices would use the theme/map system. (maybe scripts could just have url in first line)

-

EDIT2: The complete transfer system is done. I mean, it pulls the missing files if available and works with any missing resource (THEME, MAP, SCRIPT, VOICE, FLAG, GRAVE, FORT, HAT, did I miss something?)

To sum up:

Themes, maps and voices can be transferred by putting a file named "locator" in their directory, with it's first line being the URL for download.

Flags, hats, graves, forts can be transferred, by making a file with name starting with "locator" inside their respective directory. First line of the file is URL for HWP pack, then there is a list of all flags/hats/graves/forts located in that HWP (without the .png and L/R suffix)

Scripts can be transferred by putting the URL as a comment of first line in .lua file.

Clients will automatically request for missing resources and requested clients will send available URLs.

EDIT3:
About the flags/hats/forts/graves, here's a little script in Ruby that will help to create locators for them: http://pastebin.com/trsRLpnZ
(I'm assuming every Linux has Ruby, if not, I can write one in Python. Windows has no scripting interpreter by default, and batch doesn't count)
/EDIT3

TODO:
-security options I mentioned before
-warning icon for room admin to know that someone is missing resource
-proper text description for new icons
-changing team/script should send signal and update client if resource is no longer needed (currently only themes and maps do this)
-optimizations for reloading resources (right now e.g. ThemeModel updates each time new file is downloaded) (doesn't impact playing match)

I don't like the use of URLs. I would simply transfer data between the peers. Not hard to do.

Then why don't you make it?

I used URLs for two reasons:
First, game already has code for downloading from URLs, so I didn't have to "invent" anything.

Second:

Now, imagine you have a theme HWP that is over 10MB. 5 people join your room and and they all don't have it, so it means 50MB bandwidth usage. That's probably more than average hourly bandwidth and it's in few seconds.
Also, some trolls could set up false transfers for 1GB files. These could be blocked, but what would be the upper limit for the size?

Direct transfer would probably work for flags, because they are small. But I wanted the system to be consistent (that's another term for being lazy XD)

EDIT: Small update:

EDIT2: This is the icon that will appear for room admin if someone is missing a DLC:

EDIT3: Here's a simple gif presentation how it looks client side. Admin changes map to something "you" don't have so you download it and image updates.

Some minor fixes and I'm finished.

EDIT4: Forgot about options, this might take a bit~ longer.

Argh, this p2p nature of the games is sabotaging my ideas.

Following the added options, there's now unique message if download is unavailable due to restrictions:

So the system is finished.
If you can compile the dev build for Hedgewars, go to my topic to grab the code and test locators in action.

Wow. Thanks for working on this.

I do have some objections on your implementation:

- “unavailable due to restriction” this sounds too strong, as if someone else is restricting the user from doing something which is not true. It is also vague. Make it clear that this is because of an user setting.

- “A mysterious progress bar appears and then disappears. What just happened? There's a new file in your data directory.”: Make the progress bar less mysterious by adding a label on it so it is clear it is about the download.

First: Availability: I am not really happy with the locator concept. This approach only works as long the server is up AND provides the file AND at the same URL (some webmasters just LOVE to break URLs, apparently). As soon as the link is dead, the locator fails.

But honestly, I don't really understand the locator concept. Does it have to be manually added by the room starter? Or it is part of HWPs. If it is just the first, then my objection does not really make sense. xD But then I have a different objection: It would be very annoying to use for the room starter to provide a link for all files.

Second: Have you considered the case that a HWP with the exact same name is already present? It would be a shame if Hedgewars without any warning overwrites existing files.

Third: Security: This is a big one!
If you just use plain HTTP there is always the danger of a MITM attack. I would STRONGLY discourage automatic download of scripts because those are executable code. Either a Man in the Middle could modify it to malware or the server operator abuses the player's trust by including a malicious script, knowing that the players will execute these scripts on their computer. I am against making Hedgewars blindly and happily executing code from who-knows-where.

Sure, Hedgewars could also try its best to sandbox the Lua environment as much as possible but this experience shows that this tends to be error-prone. It must be a perfect sandbox, a bug can comprimise security again.

Another issue I see is with HWPs: If you automatically add HWPs to the user's directory, this invites clutter very soon. HWPs don't even need any code to break HWP. A bad HWP could for example overwrite most of the images, for example, hedgehogs start to look like worms. If HWPs gotten by the auto-downloader feature
Clearly only few would expect that this could happen after using the auto-downloader.

I think this issue could be solved by isolating ALL files gotten by the auto-downloader from the rest. If HW just drops all downloaded HWPs into Data/, this can cause serious clutter very soon, especially because HW does not provide a nice interface to manage and especially delete HWPs. Maybe HW could drop them into a seperate dir hierachy instead, only for online games and later re-use. If a player decides to keep the files from the net, manually copying to Data might be one option.

I think the question of how exactly Hedgewars handles the new files is very important in order to keep everything tidy and secure. I am not 100% sure of all this by myselves.

Graves are a minor concern. A player could make the graves completely invisible. This ruins gameplay a bit since graves are gameplay-relevant.

All other files should be more or less OK.

Forth: I would disable the lightbulb only if neccessary files are missing: Forts, maps, themes. The other files are mostly just decorations. It would be nice if the player gets these files, too, but this should be not a strict requirement.

Fifth: File size: I think for exceptionally large downloads the client should give a warning before downloading. Otherwise, a player could end up with lots of clutter in the .hedgewars directory very quickly. This issue could also be solved by only downloading some files temporarily: That is, they will only stay in RAM and a gone when exiting Hedgewars. But this would be best decided per user settings, if the user wishes to keep or not keep the files over a certain size limit.
Another user option would be to auto-reject files above another size limit, except for the important files (maps, themes, forts).

Sixth: User control: I think it is very important for the user to have much control about what gets downloaded and what not, and under which conditions. Basically I explained that in the previous point
I also suggest a setting for first prompting the user before proceeding with the download, maybe with a list of *what* will get downloaded. This prompt should be the default for large files but small files don't need a prompt IMO.
Finally, an user setting for auto-rejecting certain file types like flags or graves may also be helpful.

If I have made some false accusations, please correct me or give more explanations about the inner workings of your implementation. Overall I am still happy of course that you attempt to implement this. But I also think, if is done, it must be done *right*.

Well...

This is partially how it works. If you miss a decoration file, your light bulb is turned off only if there's a download available. If you can't get it, it's just ignored.

-

Overall, this system is still marked as "beta". What is does is just allows for transferring resources. I know there are many concerns regarding security and files and all.
It's just, the current code is pretty much copy->paste->rewrite from other frontend files, especially the downloading part comes from DLC page. The truth is, I never used QT before and I did all this just by searching in game's source for what I want and then trying to put it together somehow XD
I have no idea how to get file header from URL, that includes name and size, so filtering big files might be a problem. I also have no idea if it's possible to check from which package file comes (they are just mounted to virtual file system).

Now, why locators. I explained it before why I used them. This comes with few disadvantages, but, unless there's another option other than direct transfer between clients, it's the only reasonable one.
Seems like you are confused on how do they work. The location where you have to put the URL depends on resource type. This is because of their structure, and because, as I said, I don't really know if it's even possible to check which file comes from which package.
Anyways, it's the second option you mentioned: you normally should put them in HWPs if you want to share the HWP. E.g. for themes, it should be in the theme directory inside HWP. It of course can be outside the HWP in your Data/ directory, but if someone downloads the HWP without URL, it can't be shared further. Sharing further has few more problems, as most of hosts provide URL AFTER you upload the file, so unless you can update the file, it's impossible to upload it with link. Well... ¬_¬
Also, obviously the system fails if you can't download the file anymore. BUT, then it's just the same as it is now. You just can't play the round if the theme is missing. Also, if link is dead, no one can download the theme anyways. So direct transfer is the only solution here (server traffic), unless you re-upload the file yourself and provide new locator.

I also considered putting the downloaded files to sub-directory. But truly separating them, like, excluding from your lists until you want to use them, that would require quite a few changes. I don't actually not where are all these resources stored and how are they managed. It seems like every resource has a different way to be stored, so I modified it only when really necessary (reloading themes/maps).

You could say that if I don't know what I'm doing, I should be doing it. But then, any file transfer would be unlikely to happen. The whole subject of transferring files was on the forum since 8 years. What I made is a basic skeleton for this (which works!), with some cosmetic holes, like security concerns or potentially malicious behavior. Still better than nothing. Too bad devs are so slow in reviewing my code :/

I know you don't know how to code it yet, but I suggest system similar to Unreal engine. When you connect to the server, you download stuff to the cache directory. And cache is used only when server requests for it. Otherwise, cache is hidden from other games.

On the other hand, Source engine dumps stuff into the user directory. So you can, for example, create a server with a map that was automatically downloaded but the directory gets cluttered very easily.