Difference between revisions of "Mods repository"

From VCMI Project Wiki
Jump to: navigation, search
m (SXX moved page Mods Repository to Mods repository: title fix)
(Rework article for modders)
Line 1: Line 1:
= Problems =
+
This article will introduce you to new VCMI mod repository system and explain how new mods can be added there.
  
Current mod distribution model will eventually face following problems:
+
= How mod repository work =
  
* Lack of changes tracking repository and mods. It's not big deal now, but once we have scripting that will be very important especially if that would be Python or any other language where secure sandboxing is not possible.
+
== Where files are hosted ==
* Forward compatibility / backward compatibility. When we start to add new features newer versions of mods won't work in older versions of game. So at least we need to maintain two version of mods index: current stable and develop.
 
* We need a way to let other contributors to help us with mod maintenance and testing.
 
  
= Suggested solution =
+
Mods list hosted under main VCMI organization: [https://github.com/vcmi/vcmi-mods-repository vcmi-mods-repository].
 +
Each mod hosted in it's own repository under separate organization [https://github.com/vcmi-mods vcmi-mods]. This way if engine become more popular in future we can create separate teams for each mod and accept as many people as needed.
  
Mods index JSON hosted in git. Main branch "develop" and there will be branching into 1.00, 1.01 or whatever once they released.
+
== Why Git / GitHub? ==
  
Launcher should request specific version of JSON index from our server:
+
It's solve a lot of problems:
<pre>
+
 
https://repository.vcmi.download/develop.json
+
* Engine developers get control over all mods and can easily update them without adding extra burden for modders / mod maintainers.
https://repository.vcmi.download/0.99.json
+
* With tools such as [https://desktop.github.com/ GitHub Desktop] it's easy for non-programmers to contribute.
https://repository.vcmi.download/0.98.json
+
* Forward and backward compatibility. Stable releases of game use compatible version of mods while users of daily builds will be able to test mods supporting bleeding edge features.
</pre>
+
* Tracking of changes for repository and mods. It's not big deal now, but once we have scripting it's will be important to keep control over what code included in mods.
 +
* GitHub also create ZIP archives for us so mods will be stored uncompressed and version can be identified by commit hash.
 +
 
 +
= On backward compatibility =
  
== Mods hosting ==
+
Our mod list in vcmi-mods-repository had "develop" as primary branch. Daily builds of VCMI use mod list file from this branch.
All mods hosted in separate repositories on GitHub under separate "vcmi-mods" organization:
 
  
  https://github.com/vcmi-mods/adventure-ai-trace
+
Once VCMI get stable release there will be branching into "1.00", "1.01", etc. Launcher of released version will request mod list for particular version.
  
Trusted community members who can't help with programming can now help us to maintain mods.
+
Same way we can also create special stable branch for every mod under "vcmi-mods" organization umbrella once new stable version is released. So this way it's will be easier to maintain two versions of same mod: for stable and latest version.
  
== Mods update submission ==
+
== Redirect server ==
Once mod updated by creator he or maintainer able to make PR to index JSON. All that needed to be provided is commit hash from appropriate branch. In the end our mod repository will only contain link like that:
+
 
 +
Launcher will request specific version of JSON mod list from our server:
 
<pre>
 
<pre>
https://mods.vcmi.download/adventure-ai-trace/821190ef5396702bf763c764ed532781b1fc1ce6.zip
+
https://mods.vcmi.download/repository/develop.json
 +
https://mods.vcmi.download/repository/1.00.json
 
</pre>
 
</pre>
Since archive will be auto-generated by GitHub from commit that also provide us with somewhat secure.
 
  
== Redirect server ==
+
Our server redirect requests to github:
 +
<pre>
 +
mods.vcmi.download/repository/develop.json -> raw.githubusercontent.com/vcmi/mods-repository/develop/repository.json
 +
repository.vcmi.download/1.00.json -> raw.githubusercontent.com/vcmi/mods-repository/1.00/repository.json
 +
</pre>
  
That server just going to be redirect Launcher to proper download link like:
+
Same happen when launcher will request mods:
 
<pre>
 
<pre>
https://repository.vcmi.download/develop.json -> https://raw.githubusercontent.com/vcmi/mods-repository/develop/repository.json
+
mods.vcmi.download/archive/adventure-ai-trace/821190ef5396702bf763c764ed532781b1fc1ce6.zip -> github.com/vcmi-mods/adventure-ai-trace/archive/821190ef5396702bf763c764ed532781b1fc1ce6.zip
https://repository.vcmi.download/0.99.json -> https://raw.githubusercontent.com/vcmi/mods-repository/0.99/repository.json
 
https://mods.vcmi.download/adventure-ai-trace/821190ef5396702bf763c764ed532781b1fc1ce6.zip -> https://github.com/vcmi-mods/adventure-ai-trace/archive/821190ef5396702bf763c764ed532781b1fc1ce6.zip
 
 
</pre>
 
</pre>
So far I hope that logic can be implemented in Jekyll and hosted on GitHub Pages, but if not then we'll have something super-lightweight in Nginx Lua.
 
  
== Ideas and possible problems ==
+
This way in case of any problems with GitHub we can easily change the way mods are delivered.
There are few and likely others will appear:
 
  
* Would be great to split repository JSON into multiple small files. Sadly doubt they can be merged with Jekyll.
+
= Getting mod into repository =
* GitHub might change how ZIP created and that would change archives checksums. So if we want additional checksums over just commit hash then we need implement them different way.
 
* We should decide what organization should own mods repository (JSON) and who should be able to commit into it.
 
  
= Short FAQ =
+
== Requirements ==
 +
Right now main requirements for a mod to be accepted into VCMI repository are:
 +
* Mod must be complete. For work-in-progress mods it is better to use other way of distribution.
 +
* Mod must met some basic quality requirements. Having high-quality content is always preferable.
 +
* Mod must not contain any errors detectable by validation (console message you may see during loading)
 +
* Music files must be in Ogg/Vorbis format (*.ogg extension)
  
====What about assets licensing issues?====
+
== Getting into vcmi-mods organization ==
First of all it's only problem with some mods and they all pretty much fair use or at least not any worse than any other mod for other game. Just in case it's still better to keep them in separate organization and own repository. In worst case we'll need to host data of some particular mod separately.
 
  
====What if GitHub become less generous?====
+
To get mod into repository you need to contact person with write access to it.
It's totally possible that eventually GitHub might forbid direct RAW / ZIP access, but so far there is great amount of far more popular projects using it and VCMI unlikely ever become popular enough to reach any limits.
 
  
Even if that happen we could rapidly redirect downloads to competing services such as GitLab / BitBucket by simply keeping repository clones there. In same time infrastructure could always be on GitHub.
+
== Submitting mods to repository ==
  
====WTF is vcmi.download?====
+
Once mod ready for general public maintainer to make PR to [https://github.com/vcmi/vcmi-mods-repository vcmi-mods-repository].
That is my domain that I registered and pre-paid until end of 2026. I think it's smart to keep any gray area files outside of our main domain.
 

Revision as of 08:07, 21 February 2018

This article will introduce you to new VCMI mod repository system and explain how new mods can be added there.

How mod repository work

Where files are hosted

Mods list hosted under main VCMI organization: vcmi-mods-repository. Each mod hosted in it's own repository under separate organization vcmi-mods. This way if engine become more popular in future we can create separate teams for each mod and accept as many people as needed.

Why Git / GitHub?

It's solve a lot of problems:

  • Engine developers get control over all mods and can easily update them without adding extra burden for modders / mod maintainers.
  • With tools such as GitHub Desktop it's easy for non-programmers to contribute.
  • Forward and backward compatibility. Stable releases of game use compatible version of mods while users of daily builds will be able to test mods supporting bleeding edge features.
  • Tracking of changes for repository and mods. It's not big deal now, but once we have scripting it's will be important to keep control over what code included in mods.
  • GitHub also create ZIP archives for us so mods will be stored uncompressed and version can be identified by commit hash.

On backward compatibility

Our mod list in vcmi-mods-repository had "develop" as primary branch. Daily builds of VCMI use mod list file from this branch.

Once VCMI get stable release there will be branching into "1.00", "1.01", etc. Launcher of released version will request mod list for particular version.

Same way we can also create special stable branch for every mod under "vcmi-mods" organization umbrella once new stable version is released. So this way it's will be easier to maintain two versions of same mod: for stable and latest version.

Redirect server

Launcher will request specific version of JSON mod list from our server:

https://mods.vcmi.download/repository/develop.json
https://mods.vcmi.download/repository/1.00.json

Our server redirect requests to github:

mods.vcmi.download/repository/develop.json		-> raw.githubusercontent.com/vcmi/mods-repository/develop/repository.json
repository.vcmi.download/1.00.json		-> raw.githubusercontent.com/vcmi/mods-repository/1.00/repository.json

Same happen when launcher will request mods:

mods.vcmi.download/archive/adventure-ai-trace/821190ef5396702bf763c764ed532781b1fc1ce6.zip -> github.com/vcmi-mods/adventure-ai-trace/archive/821190ef5396702bf763c764ed532781b1fc1ce6.zip

This way in case of any problems with GitHub we can easily change the way mods are delivered.

Getting mod into repository

Requirements

Right now main requirements for a mod to be accepted into VCMI repository are:

  • Mod must be complete. For work-in-progress mods it is better to use other way of distribution.
  • Mod must met some basic quality requirements. Having high-quality content is always preferable.
  • Mod must not contain any errors detectable by validation (console message you may see during loading)
  • Music files must be in Ogg/Vorbis format (*.ogg extension)

Getting into vcmi-mods organization

To get mod into repository you need to contact person with write access to it.

Submitting mods to repository

Once mod ready for general public maintainer to make PR to vcmi-mods-repository.