Would you like to build an app with the Storj network as backend? Let's create a local test network with Storj-Sim on Windows!
If you want to try it on Linux, you can read the article https://github.com/storj/storj/wiki/Test-network
In addition, we have a article for freeBSD: Getting started with Storj Testnet on FreeNAS (freeBSD)
Install the necessary dependencies
It is recommended to install Notepad++ for easy config file editing.
- Install Go 1.14, the executable can be found here. Documentation for the installation can be found here.
- Install Msys2, which can be found here. Please, select the
Run the MSYS2 MinGW 64-bit program from the Start menu end execute:
pacman -S git make mingw-w64-x86_64-gcc
Close the MinGW terminal.
Confirm dependencies were installed correctly
Start a powershell or cmd terminal and execute:
setx path "C:\msys64\mingw64\bin;C:\msys64\usr\bin"
Please restart your terminal to apply the changes.
You should now be able to execute the following commands, please note that for each command a version number should be returned:
go version git version gcc --version (compiler of mingw)
Storj installation and setup instructions
Compile Storj from Source
To compile Storj from master, please execute the following commands:
git clone https://github.com/storj/storj -b master cd storj make install-sim
Install the REDIS binaries
We should have the redis server locally.
Download a pre-build redis binary with librarias or build your own: https://github.com/meiry/redis5_compiled_for_windows10
Copy the binary and needed libraries to the %USERPROFILE%\go\bin
Run the PostgreSQL
We can install PostgreSQL either locally, in the WSL, or in the docker container. In this example we will create a teststorj DB and will use the database user postgres.
Run the PostgreSQL in a docker container
The easiest way is to run it in the docker container, but it requires Docker installed.
PS C:\Users\USER> docker pull postgres
PS C:\Users\USER> docker run --rm -p 5432:5432 --name postgres postgres
In a new terminal, create the teststorj database:
PS C:\Users\USER> docker exec -it postgres createdb -U postgres teststorj
To run your own queries in the PostgreSQL, you can use the following command to open an interactive terminal:
PS C:\Users\USER> docker exec -it postgres psql -h localhost -U postgres teststorj
Install PostgreSQL in the WSL
If you have a WSL enabled, then you can install PostgreSQL in the Ubuntu shell.
$ sudo apt update
$ sudo apt install postgresql -y
$ sudo service postgresql start
To create teststorj database for the satellite:
$ sudo -u postgres psql
postgres=# create database teststorj;
To run your own queries in PostgreSQL you can use such command to open an interactive terminal:
$ sudo -u postgres psql teststorj
Enable access for postgres user by host
Edit the /etc/postgresql/10/main/pg_hba.conf:
$ sudo nano /etc/postgresql/10/main/pg_hba.conf
Add this line above all lines started with host:
host teststorj postgres 0.0.0.0/0 trust
Save the configuration file and restart the PostgreSQL:
$ sudo service postgresql restart
Install the native PostgreSQL locally
Setup a local Storj Network
First we have to make sure we are able to call the compiled Storj binaries. To do so, please execute the following on the command line:
setx path "C:\msys64\mingw64\bin;C:\msys64\usr\bin;%USERPROFILE%\go\bin"
Now restart the terminal and execute the following:
# This will create a local test network containing the Satellite, Uplink, S3 gateway and 10 storagenodes storj-sim network setup --postgres=postgres://postgres@localhost/teststorj?sslmode=disable # This will run the created network storj-sim network run
At the moment it's assigning ports in the following way:
- Gateways start from port
- Version control is at port
- Bootstrap server is at port
- Satellites start from port
- Satellite Console starts on port
- Storage Nodes public ports start from port
- Storage Nodes private ports start from port
Getting environment variables for the Local test network
storj-sim network env
Setup the uplink
uplink import $(storj-sim network env GATEWAY_0_ACCESS)
If the command will throw an error such as
PS C:\Users\USER> uplink import $(storj-sim network env GATEWAY_0_ACCESS)
Error: accepts between 1 and 2 arg(s), received 0
C:\Users\USER\go\bin\uplink.exe import [NAME] (ACCESS | FILE) [flags]
-h, --help help for import
--advanced if used in with -h, print advanced flags help
--config-dir string main directory for uplink configuration (default "C:\\Users\\USER\\AppData\\Roaming\\Storj\\Uplink")
Then stop the
storj-sim (Ctrl-C) and run it back (
run). After that, the configuration of uplink will work.
More info you can read at https://github.com/storj/storj/wiki/Test-network
Up- and Download Files
# This will list all buckets in your network. If the network is new, nothing should be returned. uplink ls # This is where we create bucket(mb = make bucket), e.g. uplink mb sj://Myfiles uplink mb sj://test # Now the bucket should appear in the list. uplink ls # This command is used to upload a file (cp = copy file) to your bucket. uplink cp bigfile.avi sj://test/ # This will list all files in a specific bucket. uplink ls sj://test/ # This is the command to download a file from your bucket to your machine. uplink cp sj://test/bigfile.avi bigfile.avi # This command will delete a file from a specific bucket. uplink rm sj://test/bigfile.avi
You can read more about Uplink CLI
The S3 gateway, which also is being run by
storj-sim, allows users to quickly and easily upload files to the Storj network through a S3 gateway (Minio). Furthermore, this gateway is accessible via localhost in the browser.
Copy your S3 keys from the Getting environment variables for the Local test network of
storj-sim and configure the aws CLI:
aws configure set default.aws_access_key_id eUXZt66VWTTpcwgBazQnPsuSYri aws configure set default.aws_secret_access_key xDkJKUqJVhAj69CGH1VPqDPi47Q aws configure set default.s3.multipart_threshold 1TB
Here are commands to make a bucket, upload the file, make an external link for sharing:
aws s3 --endpoint http://localhost:11000 mb s3://test3 aws s3 --endpoint http://localhost:11000 cp C:\bigvideo.avi s3://bigvideo.avi aws s3 --endpoint http://localhost:11000 ls aws s3 --endpoint http://localhost:11000 ls s3://test3 aws s3 --endpoint http://localhost:11000 presign s3://test3/bigvideo.avi
S3 gateway video streaming
Video streaming is possible with the S3 endpoint by executing the following command:
aws s3 --endpoint http://127.0.0.1:11000 mb S3://Bucket aws s3 --endpoint http://127.0.0.1:11000 cp c:\StorjIntro.mp4 S3://Bucket/StorjIntro.mp4 aws s3 --endpoint http://127.0.0.1:11000 presign s3://Bucket/StorjIntro.mp4
After the last command you will get an URL to your video file, which you can open in your browser or VLC player.
You can read more about S3 Gateway.
Relevant directories on Windows
C:\Users\USER\storj #This is where the Storj code from github is stored. C:\Users\USER\AppData\Roaming\Storj #Stores the config files. C:\Users\USER\go\bin #Stores compiled binaries.
To revert the entire installation, deleting the directories above will do the trick.