Docker Compose instructions
I am sorry to bother you again.
quick background,
For my company I used burp and burpui, absolutely loved it, even made it work with tape archive. Doing a new setup for client I moonlite for. I really wanted to use docker compose, I have had success with gitlab, Apache guacamole, this is my third docker project and I am absolutely lost in getting it work.
I am trying to do the fallowing:
OS Centos 7.7
using docker-compose file from this repo changing only the timezone: https://git.ziirish.me/ziirish/burp-ui/blob/master/docker/docker-compose.yml
I run: sudo docker-compose pull sudo docker-compose up
I receive the fallowing output
[uxadmin@backup burp]$ sudo docker-compose up
Creating network "burp_default" with the default driver
Creating burp_burp-server_1 ... done
Creating burp_pgsql_1 ... done
Creating burp_redis_1 ... done
Creating burp_burpui_1 ... done
Attaching to burp_pgsql_1, burp_redis_1, burp_burp-server_1, burp_burpui_1
pgsql_1 | The files belonging to this database system will be owned by user "postgres".
pgsql_1 | This user must also own the server process.
pgsql_1 |
pgsql_1 | The database cluster will be initialized with locale "en_US.utf8".
pgsql_1 | The default database encoding has accordingly been set to "UTF8".
pgsql_1 | The default text search configuration will be set to "english".
pgsql_1 |
pgsql_1 | Data page checksums are disabled.
pgsql_1 |
pgsql_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
pgsql_1 | creating subdirectories ... ok
pgsql_1 | selecting default max_connections ... 100
pgsql_1 | selecting default shared_buffers ... 128MB
pgsql_1 | selecting default timezone ... UTC
pgsql_1 | selecting dynamic shared memory implementation ... posix
pgsql_1 | creating configuration files ... ok
pgsql_1 | running bootstrap script ... ok
pgsql_1 | performing post-bootstrap initialization ... sh: locale: not found
pgsql_1 | 2019-11-09 07:54:06.389 UTC [26] WARNING: no usable system locales were found
pgsql_1 | ok
pgsql_1 | syncing data to disk ...
pgsql_1 | WARNING: enabling "trust" authentication for local connections
pgsql_1 | You can change this by editing pg_hba.conf or using the option -A, or
pgsql_1 | --auth-local and --auth-host, the next time you run initdb.
pgsql_1 | ok
pgsql_1 |
pgsql_1 | Success. You can now start the database server using:
pgsql_1 |
pgsql_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
redis_1 | 1:C 09 Nov 2019 07:54:06.295 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
pgsql_1 |
burp-server_1 | grep: /etc/burp/burp-server.conf: No such file or directory
redis_1 | 1:C 09 Nov 2019 07:54:06.295 # Redis version=5.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
pgsql_1 | ****************************************************
pgsql_1 | WARNING: No password has been set for the database.
redis_1 | 1:C 09 Nov 2019 07:54:06.295 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
pgsql_1 | This will allow anyone with access to the
pgsql_1 | Postgres port to access your database. In
pgsql_1 | Docker's default configuration, this is
pgsql_1 | effectively any other container on the same
pgsql_1 | system.
pgsql_1 |
pgsql_1 | Use "-e POSTGRES_PASSWORD=password" to set
pgsql_1 | it in "docker run".
pgsql_1 | ****************************************************
pgsql_1 | waiting for server to start....2019-11-09 07:54:09.071 UTC [31] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
pgsql_1 | 2019-11-09 07:54:09.125 UTC [32] LOG: database system was shut down at 2019-11-09 07:54:07 UTC
pgsql_1 | 2019-11-09 07:54:09.140 UTC [31] LOG: database system is ready to accept connections
burp-server_1 | Starting crond...
redis_1 | 1:M 09 Nov 2019 07:54:06.299 * Running mode=standalone, port=6379.
redis_1 | 1:M 09 Nov 2019 07:54:06.300 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 09 Nov 2019 07:54:06.300 # Server initialized
redis_1 | 1:M 09 Nov 2019 07:54:06.300 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 09 Nov 2019 07:54:06.301 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 09 Nov 2019 07:54:06.301 * Ready to accept connections
pgsql_1 | done
pgsql_1 | server started
pgsql_1 |
pgsql_1 | /usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/init-burpui-db.sh
pgsql_1 | CREATE ROLE
burp-server_1 | Starting burp-server...
pgsql_1 | CREATE DATABASE
pgsql_1 | GRANT
pgsql_1 |
pgsql_1 | waiting for server to shut down...2019-11-09 07:54:10.006 UTC [31] LOG: received fast shutdown request
pgsql_1 | .2019-11-09 07:54:10.009 UTC [31] LOG: aborting any active transactions
pgsql_1 | 2019-11-09 07:54:10.011 UTC [31] LOG: worker process: logical replication launcher (PID 38) exited with exit code 1
pgsql_1 | 2019-11-09 07:54:10.013 UTC [33] LOG: shutting down
pgsql_1 | 2019-11-09 07:54:10.052 UTC [31] LOG: database system is shut down
pgsql_1 | done
pgsql_1 | server stopped
pgsql_1 |
pgsql_1 | PostgreSQL init process complete; ready for start up.
pgsql_1 |
pgsql_1 | 2019-11-09 07:54:10.134 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
pgsql_1 | 2019-11-09 07:54:10.134 UTC [1] LOG: listening on IPv6 address "::", port 5432
pgsql_1 | 2019-11-09 07:54:10.149 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
pgsql_1 | 2019-11-09 07:54:10.197 UTC [42] LOG: database system was shut down at 2019-11-09 07:54:10 UTC
pgsql_1 | 2019-11-09 07:54:10.218 UTC [1] LOG: database system is ready to accept connections
burpui_1 | Setting up burp & burp-ui:
burpui_1 | bui-manage -c /etc/burp/burpui.cfg setup-burp -b /tmp/burp.conf -s /etc/burp/burp-server.conf -h burp-server -c bui -r redis:6379 -d postgresql://burpui:burpui@pgsql/burpuidb -p none
burp-server_1 | 2019-11-09 08:54:10: burp[24] could not open '/etc/burp/burp-server.conf' for reading.
burp-server_1 | 2019-11-09 08:54:11: burp[25] could not open '/etc/burp/burp-server.conf' for reading.
burp-server_1 | 2019-11-09 08:54:13: burp[27] could not open '/etc/burp/burp-server.conf' for reading.
burpui_1 | /usr/sbin/burp: unrecognized option: V
burpui_1 | Adding new option: "bconfcli=/tmp/burp.conf" to section [Burp]
burpui_1 | Unable to contact the redis server, disabling it
burpui_1 | Unable to locate burp-server configuration, aborting!
burp-server_1 | 2019-11-09 08:54:16: burp[29] could not open '/etc/burp/burp-server.conf' for reading.
burpui_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
burpui_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
burpui_1 | INFO [alembic.runtime.migration] Running upgrade -> 225d9b2f0fb1, initial
burpui_1 | INFO [alembic.runtime.migration] Running upgrade 225d9b2f0fb1 -> 7f317474332d, handle sessions
burpui_1 | INFO [alembic.runtime.migration] Running upgrade 7f317474332d -> fc07e3fa0086, sql compatibility
burpui_1 | INFO [alembic.runtime.migration] Running upgrade fc07e3fa0086 -> 56de018f4d88, user prefs
burpui_1 | Starting crond...
burpui_1 | Starting gunicorn...
burpui_1 | Starting websocket worker 1...
burpui_1 | websocket-1: started
burpui_1 | Starting websocket worker 2...
burpui_1 | websocket-2: started
burpui_1 | Starting websocket worker 3...
burpui_1 | websocket-3: started
burpui_1 | Starting websocket worker 4...
burpui_1 | websocket-4: started
burpui_1 | Starting nginx...
burpui_1 | [2019-11-09 08:54:34,260] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
burpui_1 | [2019-11-09 08:54:34,262] CRITICAL in app.create_app: Your setup is not secure! Please consider setting a secret key in your configuration file
burpui_1 | [2019-11-09 08:54:34,277] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
burpui_1 | [2019-11-09 08:54:34,278] CRITICAL in app.create_app: Your setup is not secure! Please consider setting a secret key in your configuration file
burpui_1 | [2019-11-09 08:54:34,419] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
burpui_1 | [2019-11-09 08:54:34,421] CRITICAL in app.create_app: Your setup is not secure! Please consider setting a secret key in your configuration file
burpui_1 | [2019-11-09 08:54:34,688] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
burpui_1 | [2019-11-09 08:54:34,690] CRITICAL in app.create_app: Your setup is not secure! Please consider setting a secret key in your configuration file
burpui_1 | [2019-11-09 08:54:35,055] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
burpui_1 | [2019-11-09 08:54:35,057] CRITICAL in app.create_app: Your setup is not secure! Please consider setting a secret key in your configuration file
^CGracefully stopping... (press Ctrl+C again to force)
```
``
checking /etc/burp
only "burpui.cfg" is present.
I set "appsecret" to backup (for test)
I rerun compose and see this error
> burp-server_1 | 2019-11-09 09:02:27: burp[24] could not open '/etc/burp/burp-server.conf' for reading.
so I add this config to /etc/burp:
burp-server.conf
>
```
> # This is an example config file for the burp server.
>
> mode = server
>
> # The default addresses to listen on depend upon compile time options.
> # They may be overridden here.
> address = 0.0.0.0
> port = 4971
> max_children = 5
> # Optionally configure additional ports.
> # port = 5971
> # max_children = 6
>
> # Think carefully before changing the status port address, as it can be used
> # to view the contents of backups.
> # Special value 'localhost' includes both ::1 and 127.0.0.1.
> #status_address = localhost
> # If you do not wish to run a status server at all, comment status_port out.
> status_port = 4972
> max_status_children = 15
> # Optionally configure additional status_ports.
> # status_port = 5972
> # max_status_children = 6
>
> directory = /var/spool/burp
> dedup_group = global
> clientconfdir = /etc/burp/clientconfdir
> # Choose the protocol to use.
> # 0 to decide automatically, 1 to force protocol1 mode (file level granularity
> # with a pseudo mirrored storage on the server and optional rsync). 2 forces
> # protocol2 mode (inline deduplication with variable length blocks).
> # Like many other settings, this can be set per client in the clientconfdir
> # files.
> # protocol = 0
> pidfile = /var/run/burp.server.pid
> hardlinked_archive = 0
> working_dir_recovery_method = delete
> umask = 0022
> syslog = 1
> stdout = 0
> # The following options can restrict what the client can do.
> # Restore clients can override all of these expect for force_backup.
> client_can_delete = 1
> # Set client_can_force_backup to 0 to only allow timed backups.
> client_can_force_backup = 1
> client_can_list = 1
> # Set client_can_restore to 0 if you want restores to only be initialised by
> # the server.
> client_can_restore = 1
> client_can_verify = 1
> # Ratelimit throttles the send speed. Specified in Megabits per second (Mb/s).
> # ratelimit = 1.5
> # Network timeout defaults to 7200 seconds (2 hours).
> # network_timeout = 7200
>
> # Server storage compression. Default is zlib9. Set to zlib0 to turn it off.
> #compression = zlib9
>
> # When the client version does not match the server version, log a warning.
> # Set to 0 to turn it off.
> version_warn = 1
>
> # More configuration files can be read, using syntax like the following
> # (without the leading '# ').
> # . path/to/more/conf
>
> # Location of autoupgrade files to serve to clients. Leave it commented out
> # to not autoupgrade clients.
> # autoupgrade_dir = /etc/burp/autoupgrade/server
>
> # You can have as many 'keep' lines as you like.
> # For example, if running backups daily, setting keep 7, keep 4, keep 6 will keep
> # 7 daily backups, 4 weekly, and 6 four-weekly backups.
> keep = 7
> # keep = 4
> # keep = 6
>
> # Run as different user/group.
> user=burpui
> group=burpui
>
> # CA options.
> # If you want your server to be a certificate authority and generate its own
> # certificates, uncomment the following lines. If the directory specified in
> # ca_conf does not exist, the server will create, populate it, and the paths
> # indicated by ssl_cert_ca, ssl_cert, ssl_key and ssl_dhfile below will be
> # overwritten. See docs/burp_ca.txt for more information.
> ca_conf = /etc/burp/CA.cnf
> ca_name = burpCA
> ca_server_name = backup
> ca_burp_ca = /usr/sbin/burp_ca
>
> # Check for revoked certificates in the certificate revocation list.
> # Turn this off if you use the old ssl_extra_checks_script server script.
> ca_crl_check = 1
>
> # SSL certificate authority - same file on both server and client
> ssl_cert_ca = /etc/burp/ssl_cert_ca.pem
>
> # Server SSL certificate
> ssl_cert = /etc/burp/ssl_cert-server.pem
>
> # Server SSL key
> ssl_key = /etc/burp/ssl_cert-server.key
>
> # Server SSL ciphers
> #ssl_ciphers =
>
> # Server SSL compression. Default is zlib5. Set to zlib0 to turn it off.
> #ssl_compression = zlib5
>
> # SSL key password, for loading a certificate with encryption.
> #ssl_key_password = password
>
> # Server DH file.
> ssl_dhfile = /etc/burp/dhfile.pem
>
> # The default timer_script treats the first timer_arg as the minimum interval
> # Ensure that 20 hours elapse between backups
> # Available units:
> # s (seconds), m (minutes), h (hours), d (days), w (weeks), n (months)
> timer_arg = 20h
> # Allow backups to start in the evenings and nights during weekdays
> timer_arg = Mon,Tue,Wed,Thu,Fri,00,01,02,03,04,05,19,20,21,22,23
> # Allow more hours at the weekend.
> timer_arg = Sat,Sun,00,01,02,03,04,05,06,07,08,17,18,19,20,21,22,23
> # Allow backups to start any time.
> #timer_arg = always
> # Note that, if you specify no timebands, the timer will never allow backups.
>
> # Uncomment the notify_success_* lines for email notifications of backups that
> # succeeded.
> # In the subject line, the following are substituted:
> # %b - "backup"/"restore"/"verify"
> # %c - client name
> # %w - number of warnings, if any
> #notify_success_script = /usr/share/burp/scripts/notify_script
> #notify_success_arg = sendmail -t
> #notify_success_arg = To: it_admin@domain.com
> #notify_success_arg = From: burp
> #notify_success_arg = Subject: %b succeeded: %c %w
> #notify_success_arg = Content-Type: text/plain; charset=utf-8
> # Uncomment the following to have success notifications only if there were
> # warnings.
> #notify_success_warnings_only = 1
> # Uncomment the following to have success notifications only if there were
> # new or changed files.
> #notify_success_changes_only = 1
>
> # Uncomment the following for email notifications of backups that failed.
> #notify_failure_script = /usr/share/burp/scripts/notify_script
> #notify_failure_arg = sendmail -t
> #notify_failure_arg = To: it_admin@domain.com
> #notify_failure_arg = From: burp
> #notify_failure_arg = Subject: %b failed: %c %w
> #notify_failure_arg = Content-Type: text/plain; charset=utf-8
>
> # The server can run scripts on each connection after authentication and before
> # disconnecting.
> #server_script_pre = /usr/share/burp/scripts/ssl_extra_checks_script
> #server_script_pre_arg = /etc/burp/crl
> #server_script_pre_arg = /etc/burp/burp-server.conf
> #server_script_pre_arg = /usr/share/burp/scripts/server-pre-script.local
> # Set server_script_pre_notify to 1 to have notifications on server_script_pre
> # returning non-zero. Most people will want to leave this off - it could
> # result in a lot of emails because clients normally connect once every 20
> # minutes. Requires notify_failure_script to be set above.
> #server_script_pre_notify = 0
> #server_script_post =
> #server_script_post_arg =
> #server_script_post_arg =
> #server_script_post_run_on_fail=0
> # As for server_script_pre_notify, but for post.
> #server_script_post_notify = 0
>
> # Clients that are able to list and restore files belonging to any other
> # client. If this is too permissive, you may set a restore_client for
> # individual original clients in the individual clientconfdir files.
> restore_client = bui-agent1
> # restore_client = someotherclient
>
> # Whether or not the server process should cache the tree when a monitor client
> # is browsing a backup. Advantage: speed. Disadvantage: more memory is used.
> #monitor_browse_cache = 1
> monitor_browse_cache = 1
```
I then get this error:
> burpui_1 | [2019-11-09 09:20:51,365] CRITICAL in server.load_modules: Failed loading backend for Burp version 2: [Errno 2] No such file or directory: '/etc/burp/clientconfdir'
So I add a foler named clientconfdir and set owner/group to 5337 and 775 for permissions
and now I get
burpui_1 | [2019-11-09 09:29:45,507] CRITICAL in burp2.__init__: [Errno 32] Broken pipe
I am not sure where to go from here