Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • ziirish/burp-ui
  • root42/burp-ui
  • waddles/burp-ui
  • pbrideau/burp-ui
  • malevolent/burp-ui
  • deajan/burp-ui
  • pedro.domingues/burp-ui
  • pablodav/burp-ui
8 results
Show changes
Commits on Source (274)
Hide whitespace changes
Inline Side-by-side
Showing
with 919 additions and 555 deletions
.gitignore
View file @ 8318180f
......@@ -5,9 +5,14 @@ burpui-dev.cfg*
burpui/RELEASE
devel.sh
*.egg*
.tox
.coverage
.coveragerc
.pylintrc
dist
_build
.tags
celerybeat-schedule
pkgs/burp-ui-sql/burpui_sql/VERSION
pkgs/burp-ui-extra/burpui_extra/VERSION
pkgs/burp-ui-agent/burpui_agent
.gitlab-ci.yml
View file @ 8318180f
......@@ -8,7 +8,7 @@ stages:
test:lint:
stage: test
image: ziirish/python:2.7
image: python:2.7
script:
- pip install flake8 pylint
- make flake8
......@@ -19,9 +19,10 @@ test:lint:
test:py2.7:
stage: test
image: ziirish/python:2.7
image: python:2.7
script:
- /bin/bash test/run_tests.sh
- pip install tox
- tox -e py27
tags:
- docker
except:
......@@ -29,45 +30,94 @@ test:py2.7:
test:py3.4:
stage: test
image: ziirish/python:3.4
image: python:3.4
script:
- /bin/bash test/run_tests.sh
- pip install tox
- tox -e py34
tags:
- docker
except:
- tags
- demo
test:py3.6:
stage: test
image: python:3.6
script:
- pip install tox
- tox -e py36
tags:
- docker
except:
- tags
- demo
build:py2:
stage: build
script:
- /bin/bash test/run_build.sh
- /bin/bash tests/run_build.sh
tags:
- build
only:
- master
- demo
# artifacts:
# path:
# - dist/
artifacts:
paths:
- dist/
- meta/
build:py3:
stage: build
image: ziirish/python:3.4
image: python:3.6
script:
- /bin/bash test/run_build.sh
- /bin/bash tests/run_build.sh
tags:
- build
only:
- master
# artifacts:
# paths:
# - dist/
artifacts:
paths:
- dist/
- meta/
build:docker:latest:
stage: build
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:latest -f docker/Dockerfile .
- docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:latest-py3.6 -f docker/Dockerfile-py3.6 .
- cd docker/demo/docker-pg && docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/pgsql:latest .
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:latest
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:latest-py3.6
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/pgsql:latest
tags:
- registry
only:
- rc
build:docker:release:
stage: build
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:$CI_COMMIT_TAG -f docker/Dockerfile .
- docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:$CI_COMMIT_TAG-py3.6 -f docker/Dockerfile-py3.6 .
- cd docker/demo/docker-pg && docker build -t $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/pgsql:$CI_COMMIT_TAG .
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:$CI_COMMIT_TAG
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME:$CI_COMMIT_TAG-py3.6
- docker push $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/pgsql:$CI_COMMIT_TAG
only:
- tags
tags:
- registry
deploy:demo:
stage: deploy
script:
- find docker/ -name "install" | xargs sed -i "s/@build@/$(git rev-parse HEAD)/"
- cp -r docker/ /srv/demo/
- find docker/demo/ -name "install" | xargs sed -i "s/@build@/$(git rev-parse HEAD)/"
- cd docker/demo/ && find . -maxdepth 1 -type d -a ! -name dist -exec cp -r ../../dist "{}/" \; -exec cp -r ../../meta "{}/" \; && cd ../..
- find docker/demo/ -name "Dockerfile" | xargs sed -i "s,^.*@ARTIFACTS@.*$,COPY dist/*.tar.gz /tmp/burpui.dev.tar.gz,;s,^.*@BUIAGENT_ARTIFACTS@.*$,COPY meta/burp-ui-agent*.tar.gz /tmp/burp-ui-agent.dev.tar.gz,"
- test -d /srv/demo/docker && rm -rf /srv/demo/docker
- cp -r docker/demo/ /srv/demo/docker
- cd /srv/demo/docker/
- docker-compose build
- docker-compose stop
......@@ -76,5 +126,7 @@ deploy:demo:
tags:
- deploy
only:
- rc
- demo
environment:
name: demo
url: https://demo.burp-ui.org/
.gitlab/issue_templates/Bug.md 0 → 100644
View file @ 8318180f
Hi,
You are about to submit a bug report.
First of all, make sure you are actually facing a bug.
If you have some questions about how to setup Burp-UI, make sure you read the
[doc](https://burp-ui.readthedocs.io/en/latest/) first and especially the
[FAQ](https://burp-ui.readthedocs.io/en/latest/faq.html) which already answers a
couple of questions.
Now, if you are sure you are facing a bug, please make sure to provide the
following informations:
- Bug summary
- Burp version: `burp -v`
- Burp-UI version: `burp-ui -V -v`
- Python version: `python --version`
- List the steps to reproduce your issue
- Any log that might help understand/reproduce the problem: `burp-ui -vvvv`
- Any piece of configuration that might help understand/reproduce the problem
- Any other information that you may find useful such as screenshots, etc.
**WARNING**: be sure to remove any sensitive data from your logs/configurations.
Thanks
Below is an example of a expected bug report:
----------------------------------------
Hello,
I have some trouble with Burp-UI right now. Here is a bug report:
# Bug summary
Unable to login: SQL error
# Burp
```
$ burp -v
burp-2.0.54
```
# Sysinfo
```
$ bui-manage sysinfo
Python version: 3.6.1
Burp-UI version: 0.5.0 (stable)
Single mode: True
Backend version: 2
```
# Steps to reproduce
1. Go to the login page
2. Try to authenticate
3. Authentication fail with a HTTP 500 Error
# logs
```
10.0.0.100 - - [11/Apr/2017 15:10:31] "POST /login?next=%2F HTTP/1.1" 500 -
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1994, in __call__
return self.wsgi_app(environ, start_response)
File "/opt/workspace/burp-ui/burpui/utils.py", line 412, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.6/dist-packages/flask_restplus/api.py", line 557, in error_router
return original_handler(e)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/dist-packages/flask_restplus/api.py", line 557, in error_router
return original_handler(e)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/workspace/burp-ui/burpui/routes.py", line 409, in login
user = bui.uhandler.user(form.username.data, refresh)
File "/opt/workspace/burp-ui/burpui/misc/auth/handler.py", line 52, in user
session_manager.session_expired()
File "/opt/workspace/burp-ui/burpui/sessions.py", line 39, in session_expired
return self.session_expired_by_id(self.get_session_id())
File "/opt/workspace/burp-ui/burpui/sessions.py", line 47, in session_expired_by_id
store = Session.query.filter_by(uuid=id).first()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2697, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2489, in __getitem__
return list(res)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2797, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 2820, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 469, in do_execute
cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) no such table: session [SQL: u'SELECT session.id AS session_id, session.uuid AS session_uuid, session.user AS session_user, session.ip AS session_ip, session.ua AS session_ua, session.timestamp AS session_timestamp, session.expire AS session_expire, session.permanent AS session_permanent, session.api AS session_api \nFROM session \nWHERE session.uuid = ?\n LIMIT ? OFFSET ?'] [parameters: (u'ae350427-99f4-4592-94ec-6f6a27aee59f', 1, 0)]
```
# Configuration
```
[Global]
# burp server version 1 or 2
version = 1
# Handle multiple bui-servers or not
# If set to 'false', you will need to declare at least one 'Agent' section (see
# bellow)
single = true
# authentication plugin (mandatory)
# list the misc/auth directory to see the available backends
# to disable authentication you can set "auth: none"
# you can also chain multiple backends. Example: "auth: ldap,basic"
# the order will be respected unless you manually set a higher backend priority
auth = basic, ldap
# acl plugin
# list misc/auth directory to see the available backends
# default is no ACL
acl = basic
# You can change the prefix if you are behind a reverse-proxy under a custom
# root path. For example: /burpui
prefix = none
[Production]
# storage backend (only used with gunicorn) for session and cache
# may be either 'default' or 'redis'
storage = redis
# session database to use
# may also be a backend url like: redis://localhost:6379/0
# if set to 'redis', the backend url defaults to:
# redis://<redis_host>:<redis_port>/0
# where <redis_host> is the host part, and <redis_port> is the port part of
# the below "redis" setting
session = redis
# cache database to use
# may also be a backend url like: redis://localhost:6379/0
# if set to 'redis', the backend url defaults to:
# redis://<redis_host>:<redis_port>/1
# where <redis_host> is the host part, and <redis_port> is the port part of
# the below "redis" setting
cache = redis
# redis server to connect to
redis = localhost:6379
# whether to use celery
celery = true
# database url to store some persistent data
# example: sqlite:////var/lib/burpui/store.db
database = sqlite:////tmp/burpui.db
```
Thanks
.gitlab/issue_templates/Feature Request.md 0 → 100644
View file @ 8318180f
Hi,
You are about to submit a ~"feature request".
Here are a couple of rules to follow in order to get your request approved:
- Be polite
- Provide an accurate description of what you expect
- Don't forget to add the ~"feature request" label
- Keep in mind I work on Burp-UI on my spare time so it may take some time to
get your feature request implemented
Thanks
.pylintrc deleted 100644 → 0
View file @ d19f6460
[MASTER]
# Specify a configuration file.
#rcfile=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS
# Pickle collected data for later comparisons.
persistent=yes
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
# Use multiple processes to speed up Pylint.
jobs=1
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Allow optimization of some AST trees. This will activate a peephole AST
# optimizer, which will apply various small optimizations. For instance, it can
# be used to obtain the result of joining multiple strings with the addition
# operator. Joining a lot of strings can lead to a maximum recursion error in
# Pylint and this flag can prevent that. It has one side effect, the resulting
# AST will be different than the one from reality.
optimize-ast=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time. See also the "--disable" option for examples.
#enable=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=import-star-module-level,old-octal-literal,oct-method,print-statement,unpacking-in-except,parameter-unpacking,backtick,old-raise-syntax,old-ne-operator,long-suffix,dict-view-method,dict-iter-method,metaclass-assignment,next-method-called,raising-string,indexing-exception,raw_input-builtin,long-builtin,file-builtin,execfile-builtin,coerce-builtin,cmp-builtin,buffer-builtin,basestring-builtin,apply-builtin,filter-builtin-not-iterating,line-too-long,using-cmp-argument,useless-suppression,range-builtin-not-iterating,suppressed-message,no-absolute-import,old-division,cmp-method,reload-builtin,zip-builtin-not-iterating,intern-builtin,unichr-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,input-builtin,round-builtin,hex-method,nonzero-method,map-builtin-not-iterating
[REPORTS]
# Set the output format. Available formats are text, parseable, colorized, msvs
# (visual studio) and html. You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Put messages in a separate file for each module / package specified on the
# command line instead of printing them on stdout. Reports (if any) will be
# written in a file name "pylint_global.[txt|html]".
files-output=no
# Tells whether to display a full report or only the messages
reports=yes
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,XXX,TODO
[SPELLING]
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[TYPECHECK]
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# List of classes names for which member attributes should not be checked
# (useful for classes with attributes dynamically set). This supports can work
# with qualified names.
ignored-classes=
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
[FORMAT]
# Maximum number of characters on a single line.
max-line-length=100
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
# List of optional constructs for which whitespace checking is disabled. `dict-
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
# `empty-line` allows space-only lines.
no-space-check=trailing-comma,dict-separator
# Maximum number of lines in a module
max-module-lines=1000
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
[SIMILARITIES]
# Minimum lines number of a similarity.
min-similarity-lines=4
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=no
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[VARIABLES]
# Tells whether we should check for unused import in __init__ files.
init-import=no
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_$|dummy
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,_cb
[BASIC]
# List of builtins function names that should not be used, separated by a comma
bad-functions=map,filter,input
# Good variable names which should always be accepted, separated by a comma
good-names=i,j,k,ex,Run,_
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,bar,baz,toto,tutu,tata
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Regular expression matching correct function names
function-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for function names
function-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct variable names
variable-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for variable names
variable-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct constant names
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Naming hint for constant names
const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
# Regular expression matching correct attribute names
attr-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for attribute names
attr-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct argument names
argument-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for argument names
argument-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression matching correct class attribute names
class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Naming hint for class attribute names
class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
# Regular expression matching correct inline iteration names
inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
# Naming hint for inline iteration names
inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
# Regular expression matching correct class names
class-rgx=[A-Z_][a-zA-Z0-9]+$
# Naming hint for class names
class-name-hint=[A-Z_][a-zA-Z0-9]+$
# Regular expression matching correct module names
module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Naming hint for module names
module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
# Regular expression matching correct method names
method-rgx=[a-z_][a-z0-9_]{2,30}$
# Naming hint for method names
method-name-hint=[a-z_][a-z0-9_]{2,30}$
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
[ELIF]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
[IMPORTS]
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=regsub,TERMIOS,Bastion,rexec
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,__new__,setUp
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,_fields,_replace,_source,_make
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of statements in function / method body
max-statements=50
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=Exception
CHANGELOG.rst
View file @ 8318180f
Changelog
=========
0.5.0 (09/05/2017)
------------------
- **BREAKING**: the *standalone* option has been renamed to *single* for less confusion
- **BREAKING**: the ``bui-agent`` has now its own independent package to reduce dependencies
- Add: `Spanish translation <https://git.ziirish.me/ziirish/burp-ui/merge_requests/66>`_ thanks to Pablo
- Add: `reverse_proxy option <https://git.ziirish.me/ziirish/burp-ui/merge_requests/65>`_ while running through gunicorn
- Add: `OS detection for burp 2 <https://git.ziirish.me/ziirish/burp-ui/issues/200>`_
- Add: `customizable reports <https://git.ziirish.me/ziirish/burp-ui/issues/187>`_
- Add: persistent storage for user preferences if SQL storage enabled
- Add: rate-limiting of the API
- Add: new ``diag`` and ``sysinfo`` commands to help you diagnose issues
- Improvement: the SQL storage will detect out-of-sync schemas and disable itself if needed (with a log message)
- Improvement: speedup the calendar view
- Improvement: more complete user panel (`#185 <https://git.ziirish.me/ziirish/burp-ui/issues/185>`_, `#184 <https://git.ziirish.me/ziirish/burp-ui/issues/184>`_, `#182 <https://git.ziirish.me/ziirish/burp-ui/issues/182>`_, `#132 <https://git.ziirish.me/ziirish/burp-ui/issues/132>`_)
- Fix: issue `#186 <https://git.ziirish.me/ziirish/burp-ui/issues/186>`_
- Fix: issue `#192 <https://git.ziirish.me/ziirish/burp-ui/issues/192>`_
- Fix: issue `#194 <https://git.ziirish.me/ziirish/burp-ui/issues/194>`_
- Fix: issue `#196 <https://git.ziirish.me/ziirish/burp-ui/issues/196>`_
- Fix: issue `#198 <https://git.ziirish.me/ziirish/burp-ui/issues/198>`_
- Fix: issue `#210 <https://git.ziirish.me/ziirish/burp-ui/issues/210>`_
- Various bugfix
- `Full changelog <https://git.ziirish.me/ziirish/burp-ui/compare/v0.4.0...v0.5.0>`__
0.4.4 (01/02/2017)
------------------
- Fix: issue `#193 <https://git.ziirish.me/ziirish/burp-ui/issues/193>`_
0.4.3 (12/28/2016)
------------------
- Fix: issue `#186 <https://git.ziirish.me/ziirish/burp-ui/issues/186>`_
- Fix: issue `#188 <https://git.ziirish.me/ziirish/burp-ui/issues/188>`_
- Fix: issue `#190 <https://git.ziirish.me/ziirish/burp-ui/issues/190>`_
- Fix: missing configuration in docker image
- Fix: help troubleshooting some errors
- Fix: missing vss_strip binary
- Fix: encoding error that made unable to browse backups with burp1 backend
0.4.2 (12/16/2016)
------------------
- Fix: bui-agent was broken
- Fix: handle i18n exceptions
- Fix: enable db migration only when needed
- Fix: wrong escape in translation
0.4.1 (12/15/2016)
------------------
- **BREAKING**: Use the new Flask's embedded server by default means no more SSL (HTTPS) support without a dedicated application server
- Fix: issue `#156 <https://git.ziirish.me/ziirish/burp-ui/issues/156>`_
- Fix: issue `#157 <https://git.ziirish.me/ziirish/burp-ui/issues/157>`_
- Fix: issue `#165 <https://git.ziirish.me/ziirish/burp-ui/issues/165>`_
- Fix: issue `#176 <https://git.ziirish.me/ziirish/burp-ui/issues/176>`_
- Fix: issue `#181 <https://git.ziirish.me/ziirish/burp-ui/issues/181>`_
- Fix: issue `#182 <https://git.ziirish.me/ziirish/burp-ui/issues/182>`_
- Various bugfix
- `Full changelog <https://git.ziirish.me/ziirish/burp-ui/compare/v0.4.0...v0.4.1>`__
0.4.0 (11/23/2016)
------------------
......@@ -116,7 +177,7 @@ Changelog
- Fix: issue `#99 <https://git.ziirish.me/ziirish/burp-ui/issues/99>`_
- Fix: issue `#100 <https://git.ziirish.me/ziirish/burp-ui/issues/100>`_
- Fix: issue `#101 <https://git.ziirish.me/ziirish/burp-ui/issues/101>`_
- `demo <https://demo.ziirish.me/>`_
- `demo <https://demo.burp-ui.org/>`_
- API refactoring
- Security fixes
- Bugfixes
......
CONTRIBUTORS
View file @ 8318180f
......@@ -3,10 +3,13 @@ contributed significantly to the project.
Sorted by surname (or nickname).
bedaes
Diego Daguerre
Pablo Estigarribia
Wade Fitzpatrick
Nigel Hathaway
Graham Keeling (main author of Burp)
larsen0815
Benjamin SANS (main author)
Johannes Lerch
slarti5191
Robert Tichy
Benjamin `ziirish` SANS (main author)
LICENSE
View file @ 8318180f
......@@ -4,7 +4,7 @@ The following License only applies to the burp-ui sources
================================================================================
Copyright (c) 2014-2016 by Benjamin SANS (Ziirish) <hi+burpui@ziirish.me>
Copyright (c) 2014-2017 by Benjamin SANS (Ziirish) <hi+burpui@ziirish.me>
http://ziirish.info/
Some rights reserved.
......
MANIFEST.in
View file @ 8318180f
include LICENSE
include README.rst
include CHANGELOG.rst
include MANIFEST.in
include CONTRIBUTORS
include burpui/VERSION
include burpui/RELEASE
include requirements.txt
include test-requirements.txt
include share/burpui/etc/burpui.sample.cfg
include share/burpui/etc/buiagent.sample.cfg
include contrib/debian/init.sh
include contrib/debian/bui-celery.init
include contrib/centos/init.sh
include contrib/gunicorn.d/burp-ui
include contrib/gunicorn/burpui_config.py
include bower.json
include .bowerrc
include babel.cfg
graft contrib
graft burpui
graft migrations
global-exclude *.pyc
README.rst
View file @ 8318180f
Badges
======
Burp-UI
=======
.. image:: https://git.ziirish.me/ci/projects/1/status.png?ref=stable
.. image:: https://git.ziirish.me/ziirish/burp-ui/badges/stable/build.svg
:target: https://git.ziirish.me/ziirish/burp-ui/pipelines
:alt: Build Status
......@@ -13,31 +13,32 @@ Badges
:target: https://readthedocs.org/projects/burp-ui/?badge=stable
:alt: Documentation Status
.. contents::
Introduction
============
------------
Screenshots
-----------
^^^^^^^^^^^
.. image:: https://git.ziirish.me/ziirish/burp-ui/raw/stable/docs/_static/burp-ui.gif
:target: https://git.ziirish.me/ziirish/burp-ui/blob/stable/docs/_static/burp-ui.gif
Demo
----
^^^^
A screenshot is worth a thousand words, but a Demo is worth a thousand
screenshots.
You can now play with ``Burp-UI`` at `demo.ziirish.me <https://demo.ziirish.me>`_
You can now play with ``Burp-UI`` at
`demo.burp-ui.org <https://demo.burp-ui.org/>`_
Credentials:
- *admin / admin* to play with ``Burp-UI`` as an administrator
- *demo / demo* to play with ``Burp-UI`` as a regular user
- *admin* / *admin* to play with ``Burp-UI`` as an administrator
- *demo* / *demo* to play with ``Burp-UI`` as a regular user
What's that?
------------
^^^^^^^^^^^^
Let me introduce you ``Burp-UI``. It is a web-based UI to manage your
burp-servers.
......@@ -45,11 +46,10 @@ You can view different reports about burp-servers, burp-clients, backups, etc.
``Burp-UI`` allows you to perform *online* restorations and to edit/manage
your burp-server's configuration files.
Who are you?
------------
^^^^^^^^^^^^
I'm `Ziirish <http://ziirish.info>`__, a French sysadmin who loves `Burp`_ and
I'm `Ziirish <http://ziirish.info>`__, a French *DevOps* who loves `Burp`_ and
who'd like to help its adoption by providing it a nice and powerful interface.
If you like my work, you can:
......@@ -57,45 +57,48 @@ If you like my work, you can:
* Buy me a beer or some fries (or both!)
* Make a donation on my `Paypal <http://ziirish.info>`__
Documentation
=============
-------------
The documentation is hosted on `readthedocs <https://readthedocs.org>`_ at the
following address: `burp-ui.readthedocs.io
<https://burp-ui.readthedocs.io/en/stable/>`_
following address: `burp-ui.readthedocs.io`_
FAQ
===
A `FAQ <https://burp-ui.readthedocs.io/en/stable/faq.html>`_ is available with
the documentation.
---
A `FAQ`_ is available with the documentation.
Community
=========
Please refer to the `Contributing
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_ page.
---------
Please refer to the `Contributing`_ page.
Notes
=====
-----
Feel free to report any issues on my `gitlab
<https://git.ziirish.me/ziirish/burp-ui/issues>`_.
I have closed the *github tracker* to have a unique tracker system.
Also please, read the `Contributing
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_
page before reporting any issue to make sure we have all the informations to
help you.
Also please, read the `Contributing`_ page before reporting any issue to make
sure we have all the informations to help you.
See also
--------
Starting with burp-ui v0.3.0, I introduced you `burp_server_report
<https://github.com/pablodav/burp_server_reports>`_
a project lead by Pablo Estigarribia.
Pablo also contributed to other interesting projects to automate burp and burp-ui
deployments through Ansible:
- `burpui_server <https://galaxy.ansible.com/CoffeeITWorks/burpui_server/>`_
- `burp2_server <https://galaxy.ansible.com/CoffeeITWorks/burp2_server/>`_
Licenses
========
--------
``Burp-UI`` is released under the BSD 3-clause `License`_.
......@@ -119,16 +122,15 @@ But this project is built on top of other tools. Here is a non exhaustive list:
Also note that this project is made with the Awesome `Flask`_ micro-framework.
Thanks
======
------
Thank you all for your feedbacks and bug reports. Those are making the project
moving forward.
Thank you to the `Flask`_ developers and community.
Special Thanks to Graham Keeling for its great piece of software! This project
Special Thanks to Graham Keeling for his great piece of software! This project
would not exist without `Burp`_.
......@@ -136,3 +138,6 @@ would not exist without `Burp`_.
.. _License: https://git.ziirish.me/ziirish/burp-ui/blob/stable/LICENSE
.. _Burp: http://burp.grke.org/
.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/stable/share/burpui/etc/burpui.sample.cfg
.. _burp-ui.readthedocs.io: https://burp-ui.readthedocs.io/en/stable/
.. _FAQ: https://burp-ui.readthedocs.io/en/stable/faq.html
.. _Contributing: https://burp-ui.readthedocs.io/en/stable/contributing.html
VERSION deleted 120000 → 0
View file @ d19f6460
burpui/VERSION
\ No newline at end of file
burpui/VERSION
View file @ 8318180f
0.4.0
0.5.0
burpui/__main__.py
View file @ 8318180f
......@@ -21,69 +21,96 @@ sys.path.insert(0, os.path.join(ROOT, '..'))
def parse_args(mode=True, name=None):
mname = name
if not name:
name = 'burp-ui'
parser = ArgumentParser(prog=name)
mname = 'burp-ui'
parser = ArgumentParser(prog=mname)
parser.add_argument('-v', '--verbose', dest='log', help='increase output verbosity (e.g., -vv is more verbose than -v)', action='count')
parser.add_argument('-d', '--debug', dest='debug', help='enable debug mode', action='store_true') # alias for -v
parser.add_argument('-d', '--debug', dest='debug', help='enable debug mode', action='store_true')
parser.add_argument('-V', '--version', dest='version', help='print version and exit', action='store_true')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-l', '--logfile', dest='logfile', help='output logs in defined file', metavar='<FILE>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('remaining', nargs=REMAINDER)
if mode:
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|celery|manage>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|celery|manage|legacy>')
options, unknown = parser.parse_known_args()
if mode and options.mode and options.mode not in ['celery', 'manage']:
if mode and options.mode and options.mode not in ['celery', 'manage', 'server']:
options = parser.parse_args()
unknown = []
if options.version:
from burpui.app import __title__, __version__, __release__
ver = '{}: v{}'.format(name or __title__, __version__)
from burpui.desc import __title__, __version__, __release__
ver = '{}: v{}'.format(mname or __title__, __version__)
if options.log:
ver = '{} ({})'.format(ver, __release__)
print(ver)
sys.exit(0)
return options
return options, unknown
def main():
"""
Main function
"""
options = parse_args(mode=True)
options, unknown = parse_args(mode=True)
if not options.mode or options.mode == 'server':
server(options)
server(options, unknown)
elif options.mode == 'agent':
agent(options)
elif options.mode == 'celery':
celery()
elif options.mode == 'manage':
manage()
elif options.mode == 'legacy':
legacy(options, unknown)
else:
print('Wrong mode!')
sys.exit(1)
def server(options=None):
from burpui import create_app
def server(options=None, unknown=None):
from burpui.utils import lookup_file
if unknown is None:
unknown = []
if not options:
options = parse_args(mode=False)
options, unknown = parse_args(mode=False)
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
conf = lookup_file()
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
server = create_app(conf, options.log, options.logfile, False, debug=options.debug)
if os.path.isdir('burpui'):
env['FLASK_APP'] = 'burpui/cli.py'
else:
env['FLASK_APP'] = 'burpui.cli'
env['BUI_CONFIG'] = conf
env['BUI_VERBOSE'] = str(options.log)
if options.logfile:
env['BUI_LOGFILE'] = options.logfile
if options.debug:
env['BUI_DEBUG'] = '1'
env['FLASK_DEBUG'] = '1'
env['BUI_MODE'] = 'server'
args = [
'flask',
'run'
]
args += unknown
args += [x for x in options.remaining if x != '--']
server.manual_run()
os.execvpe(args[0], args, env)
def agent(options=None):
......@@ -96,7 +123,7 @@ def agent(options=None):
patch_json()
if not options:
options = parse_args(mode=False, name='bui-agent')
options, _ = parse_args(mode=False, name='bui-agent')
conf = ['buiagent.cfg', 'buiagent.sample.cfg']
if options.config:
......@@ -114,7 +141,7 @@ def celery():
parser = ArgumentParser('bui-celery')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage|legacy>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
......@@ -127,22 +154,24 @@ def celery():
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
# make conf path absolute
if not conf.startswith('/'):
curr = os.getcwd()
conf = os.path.join(curr, conf)
check_config(conf)
os.chdir(ROOT)
env['BUI_MODE'] = 'celery'
env['BUI_CONFIG'] = conf
args = [
'celery',
'worker',
'-A',
'celery_worker.celery'
'worker.celery'
]
args += unknown
args += [x for x in options.remaining if x != '--']
......@@ -156,12 +185,15 @@ def manage():
parser = ArgumentParser('bui-manage')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage|legacy>')
parser.add_argument('-l', '--logfile', dest='logfile', help='output logs in defined file', metavar='<FILE>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
env = os.environ
if options.logfile:
env['BUI_LOGFILE'] = options.logfile
if options.config:
conf = lookup_file(options.config, guess=False)
else:
......@@ -176,6 +208,7 @@ def manage():
else:
migrations = lookup_file('migrations', directory=True)
env['BUI_MODE'] = 'manage'
env['BUI_CONFIG'] = conf
if migrations:
env['BUI_MIGRATIONS'] = migrations
......@@ -193,6 +226,47 @@ def manage():
os.execvpe(args[0], args, env)
def legacy(options=None, unknown=None):
from burpui.utils import lookup_file
if unknown is None:
unknown = []
if not options:
options, unknown = parse_args(mode=False, name='burpui-legacy')
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
env['BUI_MODE'] = 'legacy'
env['BUI_CONFIG'] = conf
if os.path.isdir('burpui'):
env['FLASK_APP'] = 'burpui/cli.py'
else:
env['FLASK_APP'] = 'burpui.cli'
env['BUI_VERBOSE'] = str(options.log)
if options.logfile:
env['BUI_LOGFILE'] = options.logfile
if options.debug:
env['BUI_DEBUG'] = '1'
env['FLASK_DEBUG'] = '1'
args = [
'flask',
'legacy'
]
args += unknown
args += [x for x in options.remaining if x != '--']
os.execvpe(args[0], args, env)
def check_config(conf):
if not conf:
raise IOError('No configuration file found')
......
burpui/_compat.py
View file @ 8318180f
......@@ -7,6 +7,7 @@
.. moduleauthor:: Ziirish <hi+burpui@ziirish.me>
"""
import re
import sys
try:
......@@ -17,9 +18,60 @@ except ImportError:
if sys.version_info[0] >= 3:
PY3 = True
from urllib.parse import unquote, quote # noqa
text_type = str
string_types = (str,)
def iterkeys(d, *args, **kwargs):
return iter(d.keys(*args, **kwargs))
def itervalues(d, *args, **kwargs):
return iter(d.values(*args, **kwargs))
def iteritems(d, *args, **kwargs):
return iter(d.items(*args, **kwargs))
def iterlists(d, *args, **kwargs):
return iter(d.lists(*args, **kwargs))
def iterlistvalues(d, *args, **kwargs):
return iter(d.listvalues(*args, **kwargs))
else:
PY3 = False
from urllib import unquote, quote # noqa
text_type = unicode
string_types = (str, unicode)
def iterkeys(d, *args, **kwargs):
return d.iterkeys(*args, **kwargs)
def itervalues(d, *args, **kwargs):
return d.itervalues(*args, **kwargs)
def iteritems(d, *args, **kwargs):
return d.iteritems(*args, **kwargs)
def iterlists(d, *args, **kwargs):
return d.iterlists(*args, **kwargs)
def iterlistvalues(d, *args, **kwargs):
return d.iterlistvalues(*args, **kwargs)
def to_bytes(text):
"""Transform string to bytes."""
if isinstance(text, text_type):
text = text.encode('utf-8')
return text
def to_unicode(input_bytes, encoding='utf-8'):
"""Decodes input_bytes to text if needed."""
if not isinstance(input_bytes, string_types):
input_bytes = input_bytes.decode(encoding)
elif re.match(r'\\u[0-9a-f]{4}', input_bytes):
input_bytes = input_bytes.decode('unicode-escape')
return input_bytes
# maps module name -> attribute name -> original item
......@@ -33,14 +85,18 @@ def patch_item(module, attr, newitem, newmodule=None):
olditem = getattr(module, attr, NONE)
if olditem is not NONE:
saved.setdefault(module.__name__, {}).setdefault(attr, olditem)
if newmodule:
if newmodule and not getattr(newmodule, 'ori_' + attr, None):
setattr(newmodule, 'ori_' + attr, olditem)
setattr(module, attr, newitem)
if not getattr(newmodule, 'ori_' + attr, None):
setattr(module, attr, newitem)
def patch_module(name, items=None):
toimport = items or []
replace_module = __import__('burpui._' + name, fromlist=toimport)
mod = __name__
if '.' in mod:
mod = mod.split('.')[0]
replace_module = __import__('{}._{}'.format(mod, name), fromlist=toimport)
module_name = name
module = __import__(module_name)
if items is None:
......
burpui/agent.py
View file @ 8318180f
......@@ -23,9 +23,10 @@ from logging.handlers import RotatingFileHandler
from .exceptions import BUIserverException
from .misc.backend.interface import BUIbackend
from ._compat import pickle
from ._compat import pickle, to_bytes, to_unicode
from .utils import BUIlogging
from .config import config
from .desc import __version__
G_PORT = 10000
G_BIND = u'::'
......@@ -52,7 +53,10 @@ class BurpHandler(BUIbackend):
self.vers = vers
self.logger = logger
module = 'burpui.misc.backend.burp{0}'.format(self.vers)
top = __name__
if '.' in top:
top = top.split('.')[0]
module = '{0}.misc.backend.burp{1}'.format(top, self.vers)
try:
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
mod = __import__(module, fromlist=['Burp'])
......@@ -165,10 +169,13 @@ class BUIAgent(BUIbackend, BUIlogging):
err = None
res = ''
lengthbuf = self.request.recv(8)
if not lengthbuf:
return
length, = struct.unpack('!Q', lengthbuf)
data = self.recvall(length)
self._logger('info', 'recv: {}'.format(data))
txt = data.decode('UTF-8')
txt = to_unicode(data)
self._logger('info', 'recv2: {}'.format(txt))
if txt == 'RE':
return
j = json.loads(txt)
......@@ -183,12 +190,14 @@ class BUIAgent(BUIbackend, BUIlogging):
res = json.dumps(getattr(parser, j['method'])(**j['args']))
else:
res = json.dumps(getattr(parser, j['method'])())
elif j['func'] == 'agent_version':
res = json.dumps(__version__)
elif j['func'] == 'restore_files':
res, err = getattr(self.client, j['func'])(**j['args'])
if err:
self.request.sendall(b'ER')
self.request.sendall(struct.pack('!Q', len(err)))
self.request.sendall(err.encode('UTF-8'))
self.request.sendall(to_bytes(err))
self._logger('error', 'Restoration failed')
return
elif j['func'] == 'get_file':
......@@ -203,7 +212,7 @@ class BUIAgent(BUIbackend, BUIlogging):
if err:
self.request.sendall(b'ER')
self.request.sendall(struct.pack('!Q', len(err)))
self.request.sendall(err.encode('UTF-8'))
self.request.sendall(to_bytes(err))
self._logger('error', err)
return
size = os.path.getsize(path)
......@@ -219,7 +228,7 @@ class BUIAgent(BUIbackend, BUIlogging):
lengthbuf = self.request.recv(8)
length, = struct.unpack('!Q', lengthbuf)
data = self.recvall(length)
txt = data.decode('UTF-8')
txt = to_unicode(data)
if txt == 'RE':
return
elif j['func'] == 'del_file':
......@@ -234,7 +243,7 @@ class BUIAgent(BUIbackend, BUIlogging):
if err:
self.request.sendall(b'ER')
self.request.sendall(struct.pack('!Q', len(err)))
self.request.sendall(err.encode('UTF-8'))
self.request.sendall(to_bytes(err))
self._logger('error', err)
return
res = json.dumps(False)
......@@ -248,14 +257,22 @@ class BUIAgent(BUIbackend, BUIlogging):
import hmac
import hashlib
from base64 import b64decode
pickles = j['args'].encode(encoding='utf-8')
pickles = to_bytes(j['args'])
key = u'{}{}'.format(self.password, j['func'])
key = key.encode(encoding='utf-8')
key = to_bytes(key)
bytes_pickles = pickles
digest = hmac.new(key, bytes_pickles, hashlib.sha1).hexdigest()
if digest != j['digest']:
raise BUIserverException('Integrity check failed: {} != {}'.format(digest, j['digest']))
j['args'] = pickle.loads(b64decode(pickles))
# We need to replace the burpui datastructure
# module by our own since it's the same but
# burpui may not be installed
mod = __name__
if '.' in mod:
mod = mod.split('.')[0]
data = b64decode(pickles)
data = data.replace(b'burpui.datastructures', to_bytes('{}.datastructures'.format(mod)))
j['args'] = pickle.loads(data)
res = json.dumps(getattr(self.client, j['func'])(**j['args']))
else:
res = json.dumps(getattr(self.client, j['func'])())
......@@ -267,10 +284,10 @@ class BUIAgent(BUIbackend, BUIlogging):
self._logger('error', traceback.format_exc())
self._logger('warning', 'Forwarding Exception: {}'.format(res))
self.request.sendall(struct.pack('!Q', len(res)))
self.request.sendall(res.encode('UTF-8'))
self.request.sendall(to_bytes(res))
return
self.request.sendall(struct.pack('!Q', len(res)))
self.request.sendall(res.encode('UTF-8'))
self.request.sendall(to_bytes(res))
except AttributeError as e:
self._logger('warning', '{}\nWrong method => {}'.format(traceback.format_exc(), str(e)))
self.request.sendall(b'KO')
......
burpui/api/__init__.py
View file @ 8318180f
......@@ -10,15 +10,18 @@
"""
import os
import sys
import uuid
import hashlib
import logging
from flask import Blueprint, Response, request, current_app
from flask import Blueprint, Response, request, current_app, session
from flask_restplus import Api as ApiPlus
from flask_login import current_user
from importlib import import_module
from functools import wraps
from .custom.namespace import Namespace
from .._compat import to_bytes
from ..server import BUIServer # noqa
from ..exceptions import BUIserverException
from ..config import config
......@@ -29,7 +32,15 @@ EXEMPT_METHODS = set(['OPTIONS'])
def cache_key():
return '{}-{}-{}'.format(current_user.get_id(), request.path, request.values)
key = '{}-{}-{}-{}-{}'.format(
session.get('login', uuid.uuid4()),
request.path,
request.values,
request.headers.get('X-Session-Tag', ''),
session.get('language', '')
)
key = hashlib.sha256(to_bytes(key)).hexdigest()
return key
def api_login_required(func):
......@@ -39,7 +50,7 @@ def api_login_required(func):
@wraps(func)
def decorated_view(*args, **kwargs):
"""decorator"""
if request.method in EXEMPT_METHODS:
if request.method in EXEMPT_METHODS: # pragma: no cover
return func(*args, **kwargs)
# 'func' is a Flask.view.MethodView so we have access to some special
# params
......@@ -71,6 +82,9 @@ class Api(ApiPlus):
CELERY_REQUIRED = ['async']
def load_all(self):
if config['WITH_LIMIT']:
from ..ext.limit import limiter
self.decorators.append(limiter.limit(config['BUI_RATIO']))
"""hack to automatically import api modules"""
if not self.loaded:
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
......@@ -81,11 +95,23 @@ class Api(ApiPlus):
ext == '.py' and
name not in ['__init__', '.', '..']):
mod = '.' + name
if name not in self.CELERY_REQUIRED or config['WITH_CELERY']:
if name not in self.CELERY_REQUIRED or \
config['WITH_CELERY']:
self.logger.debug('Loading API module: {}'.format(mod))
import_module(mod, __name__)
try:
import_module(mod, __name__)
except: # pragma: no cover
import traceback
self.logger.critical(
'Unable to load {}:\n{}'.format(
mod,
traceback.format_exc()
)
)
else:
self.logger.warning('Skipping API module: {}'.format(mod))
self.logger.warning(
'Skipping API module: {}'.format(mod)
)
def acl_admin_required(self, message='Access denied', code=403):
def decorator(func):
......@@ -101,7 +127,7 @@ class Api(ApiPlus):
def decorator(func):
@wraps(func)
def decorated(resource, *args, **kwargs):
if key not in kwargs:
if key not in kwargs: # pragma: no cover
resource.abort(500, "key '{}' not found".format(key))
if kwargs[key] != resource.username and not resource.is_admin:
resource.abort(code, message)
......@@ -114,7 +140,10 @@ class Api(ApiPlus):
@wraps(func)
def decorated(resource, *args, **kwargs):
if config['BUI_DEMO']:
resource.abort(405, 'Sorry, this feature is not available on the demo')
resource.abort(
405,
'Sorry, this feature is not available on the demo'
)
return func(resource, *args, **kwargs)
return decorated
return decorator
......@@ -130,7 +159,13 @@ class Api(ApiPlus):
apibp = Blueprint('api', __name__, url_prefix='/api')
api = Api(apibp, title='Burp-UI API', description='Burp-UI API to interact with burp', doc='/doc', decorators=[api_login_required])
api = Api(
apibp,
title='Burp-UI API',
description='Burp-UI API to interact with burp',
doc='/doc',
decorators=[api_login_required]
)
@api.errorhandler(BUIserverException)
......
burpui/api/admin.py
View file @ 8318180f
......@@ -78,9 +78,9 @@ class AuthUsers(Resource):
parser_add.add_argument('backend', required=True, help='Backend', location='values')
parser_mod = ns.parser()
parser_mod.add_argument('password', required=True, help='Password', location=('values', 'json'))
parser_mod.add_argument('backend', required=True, help='Backend', location=('values', 'json'))
parser_mod.add_argument('old_password', required=False, help='Old password', location=('values', 'json'))
parser_mod.add_argument('password', required=True, help='Password', location='values')
parser_mod.add_argument('backend', required=True, help='Backend', location='values')
parser_mod.add_argument('old_password', required=False, help='Old password', location='values')
parser_del = ns.parser()
parser_del.add_argument('backend', required=True, help='Backend', location='values')
......
burpui/api/async.py
View file @ 8318180f
......@@ -14,13 +14,17 @@ import struct
from . import api, cache_key
from .misc import History
from .custom import Resource
from .client import ClientTreeAll, node_fields
from .clients import RunningBackup, ClientsReport
from ..exceptions import BUIserverException
from ..server import BUIServer # noqa
from ..sessions import session_manager
from ..ext.async import celery
from ..ext.cache import cache
from ..ext.limit import limiter
from ..config import config
from .._compat import PY3
from ..decorators import browser_cache
from six import iteritems
from zlib import adler32
......@@ -32,6 +36,9 @@ from werkzeug.datastructures import Headers
from celery.schedules import crontab
from celery.utils.log import get_task_logger
if not PY3:
from itertools import imap as map
if config.get('WITH_SQL'):
from ..ext.sql import db
else:
......@@ -51,7 +58,7 @@ BEAT_SCHEDULE = {
},
'backup-running-minutely': {
'task': '{}.backup_running'.format(ME),
'schedule': crontab(), # run every minute
'schedule': 30.0, # run every 30 seconds
},
'get-all-backups-every-twenty-minutes': {
'task': '{}.get_all_backups'.format(ME),
......@@ -61,9 +68,9 @@ BEAT_SCHEDULE = {
'task': '{}.get_all_clients_reports'.format(ME),
'schedule': crontab(minute='*/20'), # every 20 minutes
},
'cleanup-expired-sessions-daily': {
'cleanup-expired-sessions-every-four-hours': {
'task': '{}.cleanup_expired_sessions'.format(ME),
'schedule': crontab(hour='1'), # every day at 1
'schedule': crontab(hour='*/4'), # every four hours
},
}
......@@ -135,10 +142,10 @@ def ping_backend():
except BUIserverException:
return False
map(
list(map(
__status,
iteritems(bui.client.servers)
)
))
@celery.task(bind=True)
......@@ -203,40 +210,44 @@ def cleanup_expired_sessions():
if ret:
session_manager.delete_session_by_id(sess.uuid)
return ret
map(expires, session_manager.get_expired_sessions())
list(map(expires, session_manager.get_expired_sessions()))
@celery.task
def cleanup_restore():
tasks = Task.query.filter_by(task='perform_restore').all()
tasks = db.session.query(Task).filter(Task.task == 'perform_restore').filter(datetime.utcnow() > Task.expire).all()
# tasks = Task.query.filter_by(task='perform_restore').all()
for rec in tasks:
if rec.expire and datetime.utcnow() > rec.expire:
logger.info('Task expired: {}'.format(rec))
task = perform_restore.AsyncResult(rec.uuid)
# if rec.expire and datetime.utcnow() > rec.expire:
logger.info('Task expired: {}'.format(rec))
task = perform_restore.AsyncResult(rec.uuid)
try:
if task.state != 'SUCCESS':
logger.warn(
'Task is not done yet or did not end '
'successfully: {}'.format(task.state)
)
task.revoke(terminate=True)
continue
if not task.result:
logger.warn('The task did not return anything')
continue
server = task.result.get('server')
path = task.result.get('path')
if path:
if server:
if not bui.client.del_file(path, agent=server):
logger.warn("'{}' already removed".format(path))
else:
if os.path.isfile(path):
os.unlink(path)
finally:
try:
if task.state != 'SUCCESS':
logger.warn(
'Task is not done yet or did not end '
'successfully: {}'.format(task.state)
)
task.revoke(terminate=True)
continue
if not task.result:
logger.warn('The task did not return anything')
continue
server = task.result.get('server')
path = task.result.get('path')
if path:
if server:
if not bui.client.del_file(path, agent=server):
logger.warn("'{}' already removed".format(path))
else:
if os.path.isfile(path):
os.unlink(path)
finally:
db.session.delete(rec)
db.session.commit()
task.revoke()
except:
db.session.rollback()
task.revoke()
@celery.task(bind=True)
......@@ -295,7 +306,10 @@ def perform_restore(self, client, backup,
curr = Task.query.filter_by(uuid=self.request.id).first()
if curr:
curr.expire = datetime.utcnow() + expire
db.session.commit()
try:
db.session.commit()
except:
db.session.rollback()
if err:
# make the task crash
......@@ -304,11 +318,47 @@ def perform_restore(self, client, backup,
return ret
@celery.task(bind=True)
def load_all_tree(self, client, backup, server=None, user=None):
key = 'load_all_tree-{}-{}-{}'.format(client, backup, server)
ret = cache.cache.get(key)
if ret:
return {
'client': client,
'backup': backup,
'server': server,
'user': user,
'tree': ret
}
lock_name = '{}-{}'.format(self.name, server)
# TODO: maybe do something with old_lock someday
wait_for(lock_name, self.request.id)
try:
ret = ClientTreeAll._get_tree_all(client, backup, server)
except BUIserverException as exp:
raise Exception(str(exp))
finally:
release_lock(lock_name)
cache.cache.set(key, ret, 3600)
return {
'client': client,
'backup': backup,
'server': server,
'user': user,
'tree': ret
}
def force_scheduling_now():
"""Force scheduling some tasks now"""
get_all_backups.delay()
backup_running.delay()
get_all_clients_reports.delay()
cleanup_expired_sessions.delay()
@ns.route('/status/<task_id>', endpoint='async_restore_status')
......@@ -323,6 +373,8 @@ class AsyncRestoreStatus(Resource):
This resource is part of the :mod:`burpui.api.async` module.
"""
decorators = [limiter.exempt]
@ns.doc(
responses={
200: 'Success',
......@@ -336,8 +388,11 @@ class AsyncRestoreStatus(Resource):
if db:
rec = Task.query.filter_by(uuid=task_id).first()
if rec:
db.session.delete(rec)
db.session.commit()
try:
db.session.delete(rec)
db.session.commit()
except:
db.session.rollback()
task.revoke()
err = str(task.result)
self.abort(502, err)
......@@ -402,8 +457,11 @@ class AsyncGetFile(Resource):
if db:
rec = Task.query.filter_by(uuid=task_id).first()
if rec:
db.session.delete(rec)
db.session.commit()
try:
db.session.delete(rec)
db.session.commit()
except:
db.session.rollback()
task.revoke()
if dst_server:
......@@ -598,8 +656,11 @@ class AsyncRestore(Resource):
self.username,
timedelta(minutes=60)
)
db.session.add(db_task)
db.session.commit()
try:
db.session.add(db_task)
db.session.commit()
except:
db.session.rollback()
return {'id': task.id, 'name': 'perform_restore'}, 202
......@@ -622,6 +683,7 @@ class AsyncRunningBackup(RunningBackup):
This resource is part of the :mod:`burpui.api.async` module.
"""
@api.cache.cached(timeout=60, key_prefix=cache_key)
@ns.marshal_with(
RunningBackup.running_fields,
code=200,
......@@ -731,6 +793,7 @@ class AsyncHistory(History):
403: 'Insufficient permissions',
},
)
@browser_cache(1800)
def get(self, client=None, server=None):
"""Returns a list of calendars describing the backups that have been
completed so far
......@@ -821,6 +884,7 @@ class AsyncClientsReport(ClientsReport):
500: 'Internal failure',
},
)
@browser_cache(1800)
def get(self, server=None):
"""Returns a global report about all the clients of a given server
......@@ -879,3 +943,170 @@ class AsyncClientsReport(ClientsReport):
# redirect anymore if the redirection is problematic
return redirect(url_for('api.clients_report', server=server))
return self._get_clients_reports(res, server)
@ns.route('/browseall/<name>/<int:backup>',
'/<server>/browsall/<name>/<int:backup>',
endpoint='async_client_tree_all')
@ns.doc(
params={
'server': 'Which server to collect data from when in' +
' multi-agent mode',
'name': 'Client name',
'backup': 'Backup number',
},
)
class AsyncClientTreeAll(Resource):
"""The :class:`burpui.api.async.AsyncClientTreeAll` resource allows you to
retrieve a list of all the files in a given backup through the celery
worker.
This resource is part of the :mod:`burpui.api.client` module.
An optional ``GET`` parameter called ``serverName`` is supported when
running in multi-agent mode.
"""
parser = ns.parser()
parser.add_argument(
'serverName',
help='Which server to collect data from when in multi-agent mode'
)
@ns.expect(parser)
@ns.doc(
responses={
202: 'Accepted',
405: 'Method not allowed',
403: 'Insufficient permissions',
500: 'Internal failure',
},
)
def post(self, server=None, name=None, backup=None):
"""Launch the tasks that will gather all nodes of a given backup
**POST** method provided by the webservice.
This method returns a :mod:`flask.Response` object.
:param server: Which server to collect data from when in multi-agent
mode
:type server: str
:param name: The client we are working on
:type name: str
:param backup: The backup we are working on
:type backup: int
"""
args = self.parser.parse_args()
server = server or args.get('serverName')
if not bui.client.get_attr('batch_list_supported', False, server):
self.abort(
405,
'Sorry, the requested backend does not support this method'
)
# Manage ACL
if (bui.acl and
(not bui.acl.is_client_allowed(self.username,
name,
server) and not
self.is_admin)):
self.abort(403, 'Sorry, you are not allowed to view this client')
task = load_all_tree.apply_async(
args=[
name,
backup,
server,
self.username
]
)
return {'id': task.id, 'name': 'load_all_tree'}, 202
@ns.route('/browse-status/<task_id>', endpoint='async_browse_status')
@ns.doc(
params={
'task_id': 'The task ID to process',
}
)
class AsyncBrowseStatus(Resource):
"""The :class:`burpui.api.async.AsyncBrowseStatus` resource allows you to
follow a browse task.
This resource is part of the :mod:`burpui.api.async` module.
"""
@ns.doc(
responses={
200: 'Success',
500: 'Task failed',
},
)
def get(self, task_id):
"""Returns the state of the given task"""
task = load_all_tree.AsyncResult(task_id)
if task.state == 'FAILURE':
task.revoke()
err = str(task.result)
self.abort(502, err)
if task.state == 'SUCCESS':
if not task.result:
self.abort(500, 'The task did not return anything')
server = task.result.get('server')
return {
'state': task.state,
'location': url_for(
'.async_do_browse_all',
task_id=task_id,
server=server
)
}
return {'state': task.state}
@ns.route('/get-browse/<task_id>',
'/<server>/get-browse/<task_id>',
endpoint='async_do_browse_all')
@ns.doc(
params={
'task_id': 'The task ID to process',
}
)
class AsyncDoBrowseAll(Resource):
"""The :class:`burpui.api.async.AsyncDoBrowseAll` resource allows you to
retrieve the tree generated by the given task.
This resource is part of the :mod:`burpui.api.async` module.
"""
@ns.marshal_list_with(node_fields, code=200, description='Success')
@ns.doc(
responses={
400: 'Incomplete task',
403: 'Insufficient permissions',
500: 'Task failed',
},
)
def get(self, task_id, server=None):
"""Returns the generated archive"""
task = load_all_tree.AsyncResult(task_id)
if task.state != 'SUCCESS':
if task.state == 'FAILURE':
self.abort(
500,
'Unsuccessful task: {}'.format(task.result.get('error'))
)
self.abort(400, 'Task not processed yet: {}'.format(task.state))
user = task.result.get('user')
dst_server = task.result.get('server')
resp = task.result.get('tree')
if self.username != user or (dst_server and dst_server != server):
self.abort(403, 'Unauthorized access')
task.revoke()
return resp
burpui/api/client.py
View file @ 8318180f
......@@ -428,6 +428,13 @@ class ClientTreeAll(Resource):
self.abort(403, 'Sorry, you are not allowed to view this client')
try:
json = self._get_tree_all(name, backup, server)
except BUIserverException as e:
self.abort(500, str(e))
return json
@staticmethod
def _get_tree_all(name, backup, server):
json = bui.client.get_tree(name, backup, '*', agent=server)
tree = {}
rjson = []
......@@ -487,10 +494,7 @@ class ClientTreeAll(Resource):
for fullname in roots:
rjson.append(tree[fullname])
json = rjson
except BUIserverException as e:
self.abort(500, str(e))
return json
return rjson
@ns.route('/report/<name>',
......
burpui/api/clients.py
View file @ 8318180f
......@@ -11,6 +11,7 @@ from . import api, cache_key
from ..server import BUIServer # noqa
from .custom import fields, Resource
from ..exceptions import BUIserverException
from ..decorators import browser_cache
from six import iteritems
from flask import current_app
......@@ -117,6 +118,7 @@ class RunningBackup(Resource):
'running': fields.Boolean(required=True, description='Is there a backup running right now'),
})
@api.cache.cached(timeout=60, key_prefix=cache_key)
@ns.marshal_with(running_fields, code=200, description='Success')
def get(self, server=None):
"""Tells if a backup is running right now
......@@ -191,7 +193,7 @@ class ClientsReport(Resource):
parser = ns.parser()
parser.add_argument('serverName', help='Which server to collect data from when in multi-agent mode')
parser.add_argument('limit', type=int, default=8, help='Number of elements to return')
parser.add_argument('aggregation', help='What aggregation to operate', default='number', choices=('number', 'files', 'size'))
parser.add_argument('aggregation', help='What aggregation to operate', default='number', choices=('number', 'files', 'size', 'none'))
translation = {
'number': 'number',
......@@ -202,7 +204,7 @@ class ClientsReport(Resource):
stats_fields = ns.model('ClientsStats', {
'total': fields.Integer(required=True, description='Number of files', default=0),
'totsize': fields.Integer(required=True, description='Total size occupied by all the backups of this client', default=0),
'windows': fields.String(required=True, description='Is the client a windows machine', default='unknown'),
'os': fields.String(required=True, description='OS of the client', default='unknown'),
})
client_fields = ns.model('ClientsReport', {
'name': fields.String(required=True, description='Client name'),
......@@ -251,7 +253,7 @@ class ClientsReport(Resource):
"stats": {
"total": 296377,
"totsize": 57055793698,
"windows": "unknown"
"os": "unknown"
}
},
{
......@@ -259,7 +261,7 @@ class ClientsReport(Resource):
"stats": {
"total": 3117,
"totsize": 5345361,
"windows": "true"
"os": "windows"
}
}
]
......@@ -298,6 +300,9 @@ class ClientsReport(Resource):
if limit > 1:
limit -= 1
aggregate = True
if aggregation == 'none':
aggregate = False
limit = 0
# limit the number of elements to return so the graphs stay readable
if len(backups) > limit and limit > 0:
if aggregation == 'number':
......@@ -335,18 +340,19 @@ class ClientsReport(Resource):
'stats': {
'total': 0,
'totsize': 0,
'windows': None
'os': None
}
}
# TODO: fix OS aggregation
for client in clients_orig:
if client.get('name') not in clients_name:
complement['stats']['total'] += client.get('stats', {}).get('total', 0)
complement['stats']['totsize'] += client.get('stats', {}).get('totsize', 0)
os = client.get('stats', {}).get('windows', 'unknown')
if not complement['stats']['windows']:
complement['stats']['windows'] = os
elif os != complement['stats']['windows']:
complement['stats']['windows'] = 'unknown'
os = client.get('stats', {}).get('os', 'unknown')
if not complement['stats']['os']:
complement['stats']['os'] = os
elif os != complement['stats']['os']:
complement['stats']['os'] = 'unknown'
ret['clients'].append(complement)
ret['backups'].append(backups)
......@@ -456,12 +462,12 @@ class ClientsStats(Resource):
server not in
bui.acl.servers(self.username))):
self.abort(403, 'Sorry, you don\'t have any rights on this server')
j = bui.client.get_all_clients(agent=server)
jso = bui.client.get_all_clients(agent=server)
if bui.acl and not self.is_admin:
j = [x for x in j if x['name'] in bui.acl.clients(self.username, server)]
jso = [x for x in jso if x['name'] in bui.acl.clients(self.username, server)]
except BUIserverException as e:
self.abort(500, str(e))
return j
return jso
@ns.route('/all',
......@@ -481,6 +487,7 @@ class AllClients(Resource):
"""
parser = ns.parser()
parser.add_argument('serverName', help='Which server to collect data from when in multi-agent mode')
parser.add_argument('user', help='For which user do we want the data (only works for admins')
client_fields = ns.model('AllClients', {
'name': fields.String(required=True, description='Client name'),
'agent': fields.String(required=False, default=None, description='Associated Agent name'),
......@@ -495,6 +502,7 @@ class AllClients(Resource):
403: 'Insufficient permissions',
},
)
@browser_cache(1800)
def get(self, server=None):
"""Returns a list of all clients with their associated Agent if any
......@@ -531,35 +539,44 @@ class AllClients(Resource):
ret = []
args = self.parser.parse_args()
server = server or args['serverName']
user = (args.get('user', self.username) or self.username) if \
self.is_admin else self.username
# drop privileges when switching user
if user != self.username:
self.is_admin = False
if (server and bui.acl and not self.is_admin and
server not in bui.acl.servers(self.username)):
server not in bui.acl.servers(user)):
self.abort(403, "You are not allowed to view this server infos")
if server:
clients = bui.client.get_all_clients(agent=server)
if bui.acl and not self.is_admin:
ret = [{'name': x, 'agent': server} for x in bui.acl.clients(self.username, server)]
ret = [{'name': x, 'agent': server} for x in bui.acl.clients(user, server)]
else:
ret = [{'name': x['name'], 'agent': server} for x in clients]
return ret
if bui.standalone:
if bui.acl and not self.is_admin:
ret = [{'name': x} for x in bui.acl.clients(self.username)]
ret = [{'name': x} for x in bui.acl.clients(user)]
else:
ret = [{'name': x['name']} for x in bui.client.get_all_clients()]
else:
grants = {}
if bui.acl and not self.is_admin:
for serv in bui.acl.servers(self.username):
grants[serv] = bui.acl.clients(self.username, serv)
for serv in bui.acl.servers(user):
grants[serv] = bui.acl.clients(user, serv)
else:
for serv in bui.client.servers:
grants[serv] = 'all'
for (serv, clients) in iteritems(grants):
if not isinstance(clients, list):
clients = [x['name'] for x in bui.client.get_all_clients(agent=serv)]
try:
clients = [x['name'] for x in bui.client.get_all_clients(agent=serv)]
except BUIserverException:
clients = []
ret += [{'name': x, 'agent': serv} for x in clients]
return ret