Configuration

Key	Value type	Explanation	Default value
max_links_fetching	Integer	The maximum amount of download links that shall be scraped.	(Infinite)
max_links_download	Integer	The maximum amount of files that shall be downloaded.	(Infinite)
max_filesize	Integer	Given max_filesize = x, files bigger than x MiB will be ignored.	(Infinite)
skip_resources	Integer	The number of links that shall be skipped during scraping, in other words: the offset. This is useful if the scraping process was interrupted.	0
timeout	Integer	The number of seconds after a web request fails due to a timeout.	60
storage_directory	String	The path where the files shall be saved.	./jHound
mongo_url	String	The address for the MongoDB database.	localhost
mongo_port	Integer	The port for the MongoDB database to listen to.	27017
collection_name	String	The name of the MongoDB collection where the information is stored. You can call it whatever you want.	repositories
master_address	String	The address of the master server.	localhost
master_port	Integer	The port at which the master server shall listen to.	9876
delay_seconds	Integer	The number of seconds before the master server sends a message. This is only useful for development purposes.	0
max_threads	Integer	Given max_threads = x, all nodes will use up to x threads to proceed the files.	2
nodes	Array of node objects	The node servers that shall be used.	[]

Node objects

Every “node object” is a JSON object with the following attributes (please note that address and port have to be set):

Key	Value type	Explanation	Example
address	String	The address of the node server.	192.168.2.143
port	Integer	The port of the node server.	1234
uuid	UUID4	(Optional) A UUID for the node to identify it.	ef4b378f-4428-4c82-b8ed-918787863612

Example

{
    "max_filesize": 2048,
    "timeout": 30,
    "storage_directory": "/mnt/jhound_data",
    "mongo_url": "jhound.informatik.uni-rostock.de",
    "collection_name": "repositories_uni",
    "master": {
        "address": "localhost",
        "port": 9876,
        "max_threads": 4
    },
    "nodes": [
        {
            "address": "jh1.some.domain",
            "port": 1234
        },
        {
            "address": "jh2.some.domain",
            "port": 1234
        }
    ]
}