Commits (122)
*.pyc
*.swp
*.mo
burpui-dev.cfg*
burpui/RELEASE
devel.sh
*.egg*
.coverage
.coveragerc
dist
_build
.tags
......
Changelog
=========
0.4.0 (11/23/2016)
------------------
- **BREAKING**: The database schema evolved. In order to apply these modifications, you **MUST** run the ``bui-manage db upgrade`` command after upgrading
- **BREAKING**: Plain text passwords are deprecated since v0.3.0 and are now disabled by default
- **BREAKING**: The default *version* setting has been set to ``2`` instead of ``1``
- Add: new `bui-manage setup_burp <https://git.ziirish.me/ziirish/burp-ui/merge_requests/40#note_1767>`_ command
- Add: new `docker image <https://git.ziirish.me/ziirish/burp-ui/merge_requests/40#note_1763>`_
- Add: manage `user sessions <https://git.ziirish.me/ziirish/burp-ui/merge_requests/6>`_
- Add: `French translation <https://git.ziirish.me/ziirish/burp-ui/merge_requests/4>`_
- Fix: issue `#151 <https://git.ziirish.me/ziirish/burp-ui/issues/151>`_
- Fix: issue `#154 <https://git.ziirish.me/ziirish/burp-ui/issues/154>`_
- Fix: issue `#158 <https://git.ziirish.me/ziirish/burp-ui/issues/158>`_
- Fix: issue `#163 <https://git.ziirish.me/ziirish/burp-ui/issues/163>`_
- Fix: issue `#164 <https://git.ziirish.me/ziirish/burp-ui/issues/164>`_
- Fix: issue `#166 <https://git.ziirish.me/ziirish/burp-ui/issues/166>`_
- Fix: issue `#169 <https://git.ziirish.me/ziirish/burp-ui/issues/169>`_
- Fix: issue `#171 <https://git.ziirish.me/ziirish/burp-ui/issues/171>`_
- Fix: issue `#172 <https://git.ziirish.me/ziirish/burp-ui/issues/172>`_
- Fix: issue `#173 <https://git.ziirish.me/ziirish/burp-ui/issues/173>`_
- Fix: issue `#174 <https://git.ziirish.me/ziirish/burp-ui/issues/174>`_
- Various bugfix
- `Full changelog <https://git.ziirish.me/ziirish/burp-ui/compare/v0.3.0...v0.4.0>`__
0.3.0 (08/15/2016)
------------------
......
......@@ -2,7 +2,7 @@ FROM debian:jessie
MAINTAINER hi+burpui@ziirish.me
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y supervisor logrotate locales wget curl python2.7-dev git python-virtualenv gunicorn python-pip cron libffi-dev redis-server \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y supervisor logrotate locales wget python2.7-dev git gunicorn python-pip cron libffi-dev netcat \
&& update-locale LANG=C.UTF-8 LC_MESSAGES=POSIX \
&& locale-gen en_US.UTF-8 \
&& dpkg-reconfigure -f noninteractive locales \
......@@ -21,8 +21,9 @@ RUN chmod 755 /app/setup/install
RUN /app/setup/install
EXPOSE 5000/tcp
EXPOSE 4971/tcp
EXPOSE 4972/tcp
VOLUME ["/var/spool/burp"]
VOLUME ["/etc/burp"]
ENTRYPOINT ["/app/init"]
CMD ["app:start"]
Badges
======
.. image:: https://git.ziirish.me/ci/projects/1/status.png?ref=master
:target: https://git.ziirish.me/ci/projects/1?ref=master
.. image:: https://git.ziirish.me/ci/projects/1/status.png?ref=stable
:target: https://git.ziirish.me/ziirish/burp-ui/pipelines
:alt: Build Status
.. image:: https://readthedocs.org/projects/burp-ui/badge/?version=latest
:target: https://readthedocs.org/projects/burp-ui/?badge=latest
.. image:: https://git.ziirish.me/ziirish/burp-ui/badges/stable/coverage.svg
:target: https://git.ziirish.me/ziirish/burp-ui/pipelines
:alt: Test coverage
.. image:: https://readthedocs.org/projects/burp-ui/badge/?version=stable
:target: https://readthedocs.org/projects/burp-ui/?badge=stable
:alt: Documentation Status
Introduction
......@@ -15,8 +19,8 @@ Introduction
Screenshots
-----------
.. image:: https://git.ziirish.me/ziirish/burp-ui/raw/master/docs/_static/burp-ui.gif
:target: https://git.ziirish.me/ziirish/burp-ui/blob/master/docs/_static/burp-ui.gif
.. image:: https://git.ziirish.me/ziirish/burp-ui/raw/stable/docs/_static/burp-ui.gif
:target: https://git.ziirish.me/ziirish/burp-ui/blob/stable/docs/_static/burp-ui.gif
Demo
......@@ -59,13 +63,13 @@ Documentation
The documentation is hosted on `readthedocs <https://readthedocs.org>`_ at the
following address: `burp-ui.readthedocs.io
<https://burp-ui.readthedocs.io/en/latest/>`_
<https://burp-ui.readthedocs.io/en/stable/>`_
FAQ
===
A `FAQ <https://burp-ui.readthedocs.io/en/latest/faq.html>`_ is available with
A `FAQ <https://burp-ui.readthedocs.io/en/stable/faq.html>`_ is available with
the documentation.
......@@ -73,7 +77,7 @@ Community
=========
Please refer to the `Contributing
<https://burp-ui.readthedocs.io/en/latest/contributing.html>`_ page.
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_ page.
Notes
......@@ -81,9 +85,11 @@ Notes
Feel free to report any issues on my `gitlab
<https://git.ziirish.me/ziirish/burp-ui/issues>`_.
I have closed the *github tracker* to have a unique tracker system.
Also please, read the `Contributing
<https://burp-ui.readthedocs.io/en/latest/contributing.html>`_
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_
page before reporting any issue to make sure we have all the informations to
help you.
......@@ -93,7 +99,7 @@ Licenses
``Burp-UI`` is released under the BSD 3-clause `License`_.
But this project is built on top of other tools listed here:
But this project is built on top of other tools. Here is a non exhaustive list:
- `d3.js <http://d3js.org/>`_
- `nvd3.js <http://nvd3.org/>`_
......@@ -109,7 +115,7 @@ But this project is built on top of other tools listed here:
- `AngularStrap <http://mgcrea.github.io/angular-strap/>`_
- `lodash <https://github.com/lodash/lodash>`_
- `DataTables <http://datatables.net/>`_
- Home-made `favicon <https://git.ziirish.me/ziirish/burp-ui/blob/master/burpui/static/images/favicon.ico>`_ based on pictures from `simpsoncrazy <http://www.simpsoncrazy.com/pictures/homer>`_
- Home-made `favicon <https://git.ziirish.me/ziirish/burp-ui/blob/stable/burpui/static/images/favicon.ico>`_ based on pictures from `simpsoncrazy <http://www.simpsoncrazy.com/pictures/homer>`_
Also note that this project is made with the Awesome `Flask`_ micro-framework.
......@@ -120,13 +126,13 @@ Thanks
Thank you all for your feedbacks and bug reports. Those are making the project
moving forward.
Thank you to the `Flask`_'s developers and community.
Thank you to the `Flask`_ developers and community.
Special Thanks to Graham Keeling for its great piece of software! This project
would not exist without `Burp`_.
.. _Flask: http://flask.pocoo.org/
.. _License: https://git.ziirish.me/ziirish/burp-ui/blob/master/LICENSE
.. _License: https://git.ziirish.me/ziirish/burp-ui/blob/stable/LICENSE
.. _Burp: http://burp.grke.org/
.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/master/share/burpui/etc/burpui.sample.cfg
.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/stable/share/burpui/etc/burpui.sample.cfg
[python: **.py]
encoding = utf-8
[jinja2: **/templates/**.html]
encoding = utf-8
extensions=jinja2.ext.autoescape,jinja2.ext.with_
[jinja2: **/templates/**.js]
encoding = utf-8
extensions=jinja2.ext.autoescape,jinja2.ext.with_
#!/bin/bash
python ./burpui -m agent $@
python ./burpui -m agent "$@"
#!/bin/bash
python ./burpui -m celery $@
python ./burpui -m celery "$@"
#!/bin/bash
python ./burpui -m manage $@
python ./burpui -m manage "$@"
#!/bin/bash
python ./burpui $@
python ./burpui "$@"
......@@ -9,437 +9,17 @@ jQuery/Bootstrap
.. moduleauthor:: Ziirish <hi+burpui@ziirish.me>
"""
import os
import re
import sys
import logging
import warnings
from logging import Formatter
if sys.version_info < (3, 0):
reload(sys)
sys.setdefaultencoding('utf-8')
else:
basestring = str
__title__ = 'burp-ui'
__author__ = 'Benjamin SANS (Ziirish)'
__author_email__ = 'hi+burpui@ziirish.me'
__url__ = 'https://git.ziirish.me/ziirish/burp-ui'
__doc__ = 'https://burp-ui.readthedocs.io/en/latest/'
__description__ = ('Burp-UI is a web-ui for burp backup written in python with '
'Flask and jQuery/Bootstrap')
__license__ = 'BSD 3-clause'
__version__ = open(
os.path.join(os.path.dirname(os.path.realpath(__file__)), 'VERSION')
).read().rstrip()
try: # pragma: no cover
__release__ = open(
os.path.join(os.path.dirname(os.path.realpath(__file__)), 'RELEASE')
).read().rstrip()
except: # pragma: no cover
__release__ = 'unknown'
from .app import init
warnings.simplefilter('always', RuntimeWarning)
def parse_db_setting(string):
parts = re.search(
'(?:(?P<backend>\w+)(?:\+(?P<driver>\w+))?://)?'
'(?:(?P<user>\w+)(?::?(?P<pass>.+))?@)?'
'(?P<host>[\w_.-]+):?(?P<port>\d+)?(?:/(?P<db>\w+))?',
string
)
if not parts:
raise ValueError('Unable to parse the db: "{}"'.format(string))
back = parts.group('backend') or ''
user = parts.group('user') or None
pwd = parts.group('pass') or None
host = parts.group('host') or ''
port = parts.group('port') or ''
db = parts.group('db') or ''
return (back, user, pwd, host, port, db)
def get_redis_server(myapp):
host = 'localhost'
port = 6379
if myapp.redis and myapp.redis.lower() != 'none':
try:
back, user, pwd, host, port, db = parse_db_setting(myapp.redis)
host = host or 'localhost'
try:
port = int(port)
except (ValueError, IndexError):
port = 6379
except ValueError:
pass
return host, port, pwd
def create_db(myapp):
"""Create the SQLAlchemy instance if possible
:param myapp: Application context
:type myapp: :class:`burpui.server.BUIServer`
"""
if myapp.config['WITH_SQL']:
from .ext.sql import db
myapp.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db.init_app(myapp)
return db
return None
def create_celery(myapp, warn=True):
"""Create the Celery app if possible
:param myapp: Application context
:type myapp: :class:`burpui.server.BUIServer`
"""
if myapp.config['WITH_CELERY']:
from .ext.async import celery
host, oport, pwd = get_redis_server(myapp)
odb = 2
if isinstance(myapp.use_celery, basestring):
try:
(_, _, pwd, host, port, db) = parse_db_setting(myapp.use_celery)
if not port:
port = oport
if not db:
db = odb
else:
try:
db = int(db)
except ValueError:
db = odb
except ValueError:
pass
else:
db = odb
port = oport
if pwd:
redis_url = 'redis://:{}@{}:{}/{}'.format(pwd, host, port, db)
else:
redis_url = 'redis://{}:{}/{}'.format(host, port, db)
myapp.config['CELERY_BROKER_URL'] = myapp.config['BROKER_URL'] = \
redis_url
myapp.config['CELERY_RESULT_BACKEND'] = redis_url
celery.conf.update(myapp.config)
if not hasattr(celery, 'flask_app'):
celery.flask_app = myapp
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with myapp.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
return celery
if warn:
message = 'Something went wrong while initializing celery worker.\n' \
'Maybe it is not enabled in your conf ' \
'({}).'.format(myapp.config['CFG'])
warnings.warn(
message,
RuntimeWarning
)
return None
def init(conf=None, verbose=0, logfile=None, gunicorn=True, unittest=False, debug=False):
"""Initialize the whole application.
:param conf: Configuration file to use
:type conf: str
:param verbose: Set the verbosity level
:type verbose: int
:param logfile: Store the logs in the given file
:type logfile: str
:param gunicorn: Enable gunicorn engine instead of flask's default
:type gunicorn: bool
:param unittest: Are we running tests (used for test only)
:type unittest: bool
:param debug: Enable debug mode
:type debug: bool
:returns: A :class:`burpui.server.BUIServer` object
"""
from flask_login import LoginManager
from flask_bower import Bower
from .utils import basic_login_from_request, ReverseProxied, lookup_file
from .server import BUIServer as BurpUI
from .routes import view
from .api import api, apibp
from .ext.cache import cache
logger = logging.getLogger('burp-ui')
# The debug argument used to be a boolean so we keep supporting this format
if isinstance(verbose, bool):
if verbose:
verbose = logging.DEBUG
else:
verbose = logging.CRITICAL
else:
levels = [
logging.CRITICAL,
logging.ERROR,
logging.WARNING,
logging.INFO,
logging.DEBUG
]
if verbose >= len(levels):
verbose = len(levels) - 1
if not verbose:
verbose = 0
verbose = levels[verbose]
if logfile:
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
logfile,
maxBytes=1024 * 1024 * 100,
backupCount=5
)
else:
from logging import StreamHandler
handler = StreamHandler()
if verbose > logging.DEBUG:
LOG_FORMAT = (
'[%(asctime)s] %(levelname)s in '
'%(module)s.%(funcName)s: %(message)s'
)
else:
LOG_FORMAT = (
'-' * 80 + '\n' +
'%(levelname)s in %(module)s.%(funcName)s ' +
'[%(pathname)s:%(lineno)d]:\n' +
'%(message)s\n' +
'-' * 80
)
handler.setLevel(verbose)
handler.setFormatter(Formatter(LOG_FORMAT))
logger.setLevel(verbose)
logger.addHandler(handler)
logger.debug(
'conf: {}\n'.format(conf) +
'verbose: {}\n'.format(logging.getLevelName(verbose)) +
'logfile: {}\n'.format(logfile) +
'gunicorn: {}\n'.format(gunicorn) +
'debug: {}\n'.format(debug) +
'unittest: {}'.format(unittest)
)
if not unittest:
from ._compat import patch_json
patch_json()
# We initialize the core
app = BurpUI()
if verbose:
app.enable_logger()
app.gunicorn = gunicorn
app.config['CFG'] = None
# FIXME: strange behavior when bundling errors
# app.config['BUNDLE_ERRORS'] = True
app.config['REMEMBER_COOKIE_HTTPONLY'] = True
if debug and not gunicorn: # pragma: no cover
app.config['DEBUG'] = True and not unittest
app.config['TESTING'] = True and not unittest
# Still need to test conf file here because the init function can be called
# by gunicorn directly
if conf:
app.config['CFG'] = lookup_file(conf, guess=False)
else:
app.config['CFG'] = lookup_file()
logger.info('Using configuration: {}'.format(app.config['CFG']))
app.setup(app.config['CFG'])
if debug:
app.config.setdefault('TEMPLATES_AUTO_RELOAD', True)
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.jinja_env.globals.update(
isinstance=isinstance,
list=list,
version_id='{}-{}'.format(__version__, __release__),
config=app.config
)
# manage application secret key
if app.secret_key and (app.secret_key.lower() == 'none' or
(app.secret_key.lower() == 'random' and gunicorn)):
logger.warning('Your setup is not secure! Please consider setting a'
' secret key in your configuration file')
app.secret_key = 'Burp-UI'
if not app.secret_key or app.secret_key.lower() == 'random':
from base64 import b64encode
app.secret_key = b64encode(os.urandom(256))
app.wsgi_app = ReverseProxied(app.wsgi_app, app)
# Manage gunicorn special tricks & improvements
if gunicorn: # pragma: no cover
logger.info('Using gunicorn')
from werkzeug.contrib.fixers import ProxyFix
app.wsgi_app = ProxyFix(app.wsgi_app)
if app.storage and app.storage.lower() != 'default':
try:
# Session setup
if not app.session_db or app.session_db.lower() != 'none':
from redis import Redis
from flask_session import Session
host, port, pwd = get_redis_server(app)
db = 0
if app.session_db and app.session_db.lower() != 'default':
try:
(_, _, pwd, host, port, db) = \
parse_db_setting(app.session_db)
except ValueError as exp:
logger.warning(str(exp))
try:
db = int(db)
except ValueError:
db = 0
logger.debug('Using redis://guest:****@{}:{}/{}'.format(
host,
port,
db)
)
red = Redis(host=host, port=port, db=db, password=pwd)
app.config['SESSION_TYPE'] = 'redis'
app.config['SESSION_REDIS'] = red
app.config['SESSION_USE_SIGNER'] = app.secret_key is not None
app.config['SESSION_PERMANENT'] = False
ses = Session()
ses.init_app(app)
# Cache setup
if not app.cache_db or app.cache_db.lower() != 'none':
host, port, pwd = get_redis_server(app)
db = 1
if app.cache_db and app.cache_db.lower() != 'default':
try:
(_, _, pwd, host, port, db) = \
parse_db_setting(app.cache_db)
except ValueError as exp:
logger.warning(str(exp))
try:
db = int(db)
except ValueError:
db = 1
logger.debug('Using redis://guest:****@{}:{}/{}'.format(
host,
port,
db)
)
cache.init_app(
app,
config={
'CACHE_TYPE': 'redis',
'CACHE_REDIS_HOST': host,
'CACHE_REDIS_PORT': port,
'CACHE_REDIS_PASSWORD': pwd,
'CACHE_REDIS_DB': db
}
)
# clear cache at startup in case we removed or added servers
with app.app_context():
cache.clear()
else:
cache.init_app(app)
except Exception as e:
logger.warning('Unable to initialize redis: {}'.format(str(e)))
cache.init_app(app)
else:
cache.init_app(app)
# Create celery app if enabled
create_celery(app, warn=False)
# Create SQLAlchemy if enabled
create_db(app)
# We initialize the API
api.version = __version__
api.release = __release__
api.__url__ = __url__
api.__doc__ = __doc__
api.load_all()
app.register_blueprint(apibp)
# Then we load our routes
view.__url__ = __url__
view.__doc__ = __doc__
app.register_blueprint(view)
# And the login_manager
app.login_manager = LoginManager()
app.login_manager.login_view = 'view.login'
app.login_manager.login_message_category = 'info'
app.login_manager.session_protection = 'strong'
app.login_manager.init_app(app)
app.config.setdefault(
'BOWER_COMPONENTS_ROOT',
os.path.join('static', 'vendor')
)
app.config.setdefault('BOWER_REPLACE_URL_FOR', True)
bower = Bower()
bower.init_app(app)
@app.before_request
def setup_request():
# make sure to store secure cookie if required
if app.scookie:
from flask import request
criteria = [
request.is_secure,
request.headers.get('X-Forwarded-Proto', 'http') == 'https'
]
app.config['SESSION_COOKIE_SECURE'] = \
app.config['REMEMBER_COOKIE_SECURE'] = any(criteria)
@app.login_manager.user_loader
def load_user(userid):
"""User loader callback"""
if app.auth != 'none':
return app.uhandler.user(userid)
return None
@app.login_manager.request_loader
def load_user_from_request(request):
"""User loader from request callback"""
if app.auth != 'none':
return basic_login_from_request(request, app)
return app
if sys.version_info < (3, 0): # pragma: no cover
reload(sys)
sys.setdefaultencoding('utf-8')
# backward compatibility
create_app = init
......@@ -27,7 +27,7 @@ def parse_args(mode=True, name=None):
parser.add_argument('-v', '--verbose', dest='log', help='increase output verbosity (e.g., -vv is more verbose than -v)', action='count')
parser.add_argument('-d', '--debug', dest='debug', help='enable debug mode', action='store_true') # alias for -v
parser.add_argument('-V', '--version', dest='version', help='print version and exit', action='store_true')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-l', '--logfile', dest='logfile', help='output logs in defined file', metavar='<FILE>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('remaining', nargs=REMAINDER)
......@@ -39,7 +39,7 @@ def parse_args(mode=True, name=None):
options = parser.parse_args()
if options.version:
from burpui import __title__, __version__, __release__
from burpui.app import __title__, __version__, __release__
ver = '{}: v{}'.format(name or __title__, __version__)
if options.log:
ver = '{} ({})'.format(ver, __release__)
......@@ -113,16 +113,20 @@ def celery():
from burpui.utils import lookup_file
parser = ArgumentParser('bui-celery')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
conf = lookup_file()
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
# make conf path absolute
......@@ -132,7 +136,6 @@ def celery():
os.chdir(ROOT)
env = os.environ
env['BUI_CONFIG'] = conf
args = [
......@@ -151,17 +154,21 @@ def manage():
from burpui.utils import lookup_file
parser = ArgumentParser('bui-manage')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
conf = lookup_file()
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
if options.migrations:
......@@ -169,14 +176,16 @@ def manage():
else:
migrations = lookup_file('migrations', directory=True)
env = os.environ
env['BUI_CONFIG'] = conf
if migrations:
env['BUI_MIGRATIONS'] = migrations
if os.path.isdir('burpui'):
env['FLASK_APP'] = 'burpui/cli.py'
else:
env['FLASK_APP'] = 'burpui.cli'
args = [
sys.executable,
os.path.join(ROOT, 'manage.py'),
'flask'
]
args += unknown
args += [x for x in options.remaining if x != '--']
......
......@@ -12,4 +12,6 @@
from . import create_app
app = create_app(conf='/dev/null', gunicorn=False)
# This is a lie we are not really unittesting, but we want to avoid the v2
# errors
app = create_app(conf='/dev/null', gunicorn=False, unittest=True)
......@@ -30,7 +30,7 @@ from .config import config
G_PORT = 10000
G_BIND = u'::'
G_SSL = False
G_VERSION = 1
G_VERSION = 2
G_SSLCERT = u''
G_SSLKEY = u''
G_PASSWORD = u'password'
......@@ -48,7 +48,7 @@ class BurpHandler(BUIbackend):
foreign = BUIbackend.__abstractmethods__
BUIbackend.__abstractmethods__ = frozenset()
def __init__(self, vers=1, logger=None, conf=None):
def __init__(self, vers=2, logger=None, conf=None):
self.vers = vers
self.logger = logger
......@@ -95,6 +95,7 @@ class BUIAgent(BUIbackend, BUIlogging):
def __init__(self, conf=None, level=0, logfile=None, debug=False):
self.debug = debug
self.padding = 1
level = level or 0
if level > logging.NOTSET:
logging.addLevelName(DISCLOSURE, 'DISCLOSURE')
levels = [
......@@ -141,8 +142,9 @@ class BUIAgent(BUIbackend, BUIlogging):
self.sslcert = self.conf.safe_get('sslcert')
self.sslkey = self.conf.safe_get('sslkey')
self.password = self.conf.safe_get('password')
self.conf.setdefault('BUI_AGENT', True)
self.cli = BurpHandler(self.vers, self.logger, self.conf)
self.client = BurpHandler(self.vers, self.logger, self.conf)
pool = Pool(10000)
if not self.ssl:
self.server = StreamServer((self.bind, self.port), self.handle, spawn=pool)
......@@ -176,13 +178,13 @@ class BUIAgent(BUIbackend, BUIlogging):
return
try:
if j['func'] == 'proxy_parser':
parser = self.cli.get_parser()
parser = self.client.get_parser()
if j['args']:
res = json.dumps(getattr(parser, j['method'])(**j['args']))
else:
res = json.dumps(getattr(parser, j['method'])())
elif j['func'] == 'restore_files':
res, err = getattr(self.cli, j['func'])(**j['args'])
res, err = getattr(self.client, j['func'])(**j['args'])
if err:
self.request.sendall(b'ER')
self.request.sendall(struct.pack('!Q', len(err)))
......@@ -195,7 +197,7 @@ class BUIAgent(BUIbackend, BUIlogging):
err = None
if not path.startswith('/'):
err = 'The path must be absolute! ({})'.format(path)
if not path.startswith(self.cli.tmpdir):
if not path.startswith(self.client.tmpdir):
err = 'You are not allowed to access this path: ' \
'({})'.format(path)
if err:
......@@ -226,7 +228,7 @@ class BUIAgent(BUIbackend, BUIlogging):
err = None
if not path.startswith('/'):
err = 'The path must be absolute! ({})'.format(path)
if not path.startswith(self.cli.tmpdir):
if not path.startswith(self.client.tmpdir):
err = 'You are not allowed to access this path: ' \
'({})'.format(path)
if err:
......@@ -246,26 +248,24 @@ class BUIAgent(BUIbackend, BUIlogging):
import hmac
import hashlib
from base64 import b64decode
pickles = j['args']
pickles = j['args'].encode(encoding='utf-8')
key = u'{}{}'.format(self.password, j['func'])
key = key.encode(encoding='utf-8')
bytes_pickles = pickles.encode(encoding='utf-8')
bytes_pickles = pickles
digest = hmac.new(key, bytes_pickles, hashlib.sha1).hexdigest()
if digest != j['digest']:
raise BUIserverException('Integrity check failed')
raise BUIserverException('Integrity check failed: {} != {}'.format(digest, j['digest']))
j['args'] = pickle.loads(b64decode(pickles))
res = json.dumps(getattr(self.cli, j['func'])(**j['args']))
res = json.dumps(getattr(self.client, j['func'])(**j['args']))
else:
res = json.dumps(getattr(self.cli, j['func'])())
res = json.dumps(getattr(self.client, j['func'])())
self._logger('info', 'result: {}'.format(res))
self.request.sendall(b'OK')
# should not happen
except Exception as e:
raise BUIserverException(str(e))
except BUIserverException as e:
except (BUIserverException, Exception) as e:
self.request.sendall(b'ER')
res = str(e)
self._logger('error', 'Forwarding Exception: {}'.format(res))
self._logger('error', traceback.format_exc())
self._logger('warning', 'Forwarding Exception: {}'.format(res))
self.request.sendall(struct.pack('!Q', len(res)))
self.request.sendall(res.encode('UTF-8'))
return
......
......@@ -41,12 +41,12 @@ def api_login_required(func):
"""decorator"""
if request.method in EXEMPT_METHODS:
return func(*args, **kwargs)
try:
name = func.func_name
except: # pragma: no cover
name = func.__name__
# 'func' is a Flask.view.MethodView so we have access to some special
# params
cls = func.view_class
login_required = getattr(cls, 'login_required', True)
if (bui.auth != 'none' and
name not in api.LOGIN_NOT_REQUIRED and
login_required and
not bui.config.get('LOGIN_DISABLED', False)):
if not current_user.is_authenticated:
if request.headers.get('X-From-UI', False):
......@@ -68,7 +68,6 @@ class Api(ApiPlus):
release = None
__doc__ = None
__url__ = None
LOGIN_NOT_REQUIRED = []
CELERY_REQUIRED = ['async']
def load_all(self):
......@@ -110,6 +109,16 @@ class Api(ApiPlus):
return decorated
return decorator
def disabled_on_demo(self):
def decorator(func):
@wraps(func)
def decorated(resource, *args, **kwargs):
if config['BUI_DEMO']:
resource.abort(405, 'Sorry, this feature is not available on the demo')
return func(resource, *args, **kwargs)
return decorated
return decorator
def namespace(self, *args, **kwargs):
"""A namespace factory
......
......@@ -9,6 +9,8 @@
"""
from . import api
from ..server import BUIServer # noqa
from ..sessions import session_manager
from ..utils import NOTIF_OK
from .custom import fields, Resource
# from ..exceptions import BUIserverException
......@@ -24,6 +26,16 @@ user_fields = ns.model('Users', {
'name': fields.String(required=True, description='User name'),
'backend': fields.String(required=True, description='Backend name'),
})
session_fields = ns.model('Sessions', {
'uuid': fields.String(description='Session id'),
'ip': fields.String(description='IP address'),
'ua': fields.String(description='User-Agent'),
'permanent': fields.Boolean(description='Remember cookie'),
'api': fields.Boolean(description='API login'),
'expire': fields.DateTime(description='Expiration date'),
'timestamp': fields.DateTime(description='Last seen'),
'current': fields.Boolean(description='Is current session', default=False)
})
@ns.route('/me', endpoint='admin_me')
......@@ -68,6 +80,7 @@ class AuthUsers(Resource):
parser_mod = ns.parser()
parser_mod.add_argument('password', required=True, help='Password', location=('values', 'json'))
parser_mod.add_argument('backend', required=True, help='Backend', location=('values', 'json'))
parser_mod.add_argument('old_password', required=False, help='Old password', location=('values', 'json'))
parser_del = ns.parser()
parser_del.add_argument('backend', required=True, help='Backend', location='values')
......@@ -118,6 +131,7 @@ class AuthUsers(Resource):
})
return ret
@api.disabled_on_demo()
@api.acl_admin_required(message="Not allowed to create users")
@ns.expect(parser_add)
@ns.doc(
......@@ -159,6 +173,7 @@ class AuthUsers(Resource):
status = 201 if success else 200
return [[code, message]], status
@api.disabled_on_demo()
@api.acl_admin_required(message="Not allowed to delete this user")
@ns.expect(parser_del)
@ns.doc(
......@@ -199,6 +214,7 @@ class AuthUsers(Resource):
status = 201 if success else 200
return [[code, message]], status
@api.disabled_on_demo()
@api.acl_own_or_admin(key='name', message="Not allowed to modify this user")
@ns.expect(parser_mod)
@ns.doc(
......@@ -215,6 +231,9 @@ class AuthUsers(Resource):
"""Change user password"""
args = self.parser_mod.parse_args()
if not self.is_admin and not args['old_password']:
self.abort(400, "Old password required")
try:
handler = getattr(bui, 'uhandler')
except AttributeError:
......@@ -235,7 +254,8 @@ class AuthUsers(Resource):
success, message, code = backend.change_password(
name,
args['password']
args['password'],
args.get('old_password')
)
status = 201 if success else 200
return [[code, message]], status
......@@ -288,3 +308,68 @@ class AuthBackends(Resource):
})
return ret
@ns.route('/me/session',
'/me/session/<id>',
endpoint='user_sessions')
@ns.doc(
params={
'id': 'Session id',
}
)
class MySessions(Resource):
"""The :class:`burpui.api.admin.MySessions` resource allows you to
retrieve a list of sessions and invalidate them for the current user.
This resource is part of the :mod:`burpui.api.admin` module.
"""
@ns.marshal_list_with(session_fields, code=200, description='Success')
@ns.doc(
responses={
403: 'Insufficient permissions',
404: 'User not found',
},
)
def get(self, id=None):
"""Returns a list of sessions
**GET** method provided by the webservice.
:returns: Sessions
"""
if id:
return session_manager.get_session_by_id(id)
user = getattr(current_user, 'name', None)
if not user:
self.abort(404, 'User not found')
return session_manager.get_user_sessions(user)
@api.disabled_on_demo()
@ns.doc(
responses={
201: 'Success',
403: 'Insufficient permissions',
404: 'User or session not found',
400: 'Wrong request'
}
)
def delete(self, id=None):
"""Delete a given session
Note: ``id`` is mandatory
"""
if not id:
self.abort(400, 'Missing id')
user = getattr(current_user, 'name', None)
if not user:
self.abort(404, 'User not found')
store = session_manager.get_session_by_id(id)
if not store:
self.abort('Session not found')
if store.user != user:
self.abort(403, 'Insufficient permissions')
if session_manager.invalidate_session_by_id(store.uuid):
session_manager.delete_session_by_id(store.uuid)
return [NOTIF_OK, 'Session {} successfully revoked'.format(id)], 201
......@@ -12,14 +12,15 @@ import select
import struct
from . import api, cache_key
from .clients import RunningBackup
from .misc import History
from ..server import BUIServer # noqa
from .custom import Resource
from .clients import RunningBackup, ClientsReport
from ..exceptions import BUIserverException
from ..server import BUIServer # noqa
from ..sessions import session_manager
from ..ext.async import celery
from ..ext.cache import cache
from ..config import config
from ..exceptions import BUIserverException
from six import iteritems
from zlib import adler32
......@@ -52,9 +53,17 @@ BEAT_SCHEDULE = {
'task': '{}.backup_running'.format(ME),
'schedule': crontab(), # run every minute
},
'get-all-backups-bi-hourly': {
'get-all-backups-every-twenty-minutes': {
'task': '{}.get_all_backups'.format(ME),
'schedule': crontab(minute='23,53'), # every 30 minutes
'schedule': crontab(minute='*/20'), # every 20 minutes
},
'get-all-clients-reports-every-twenty-minutes': {
'task': '{}.get_all_clients_reports'.format(ME),
'schedule': crontab(minute='*/20'), # every 20 minutes
},
'cleanup-expired-sessions-daily': {
'task': '{}.cleanup_expired_sessions'.format(ME),
'schedule': crontab(hour='1'), # every day at 1
},
}
......@@ -76,19 +85,24 @@ else:
def acquire_lock(name, value='nyan', timeout=LOCK_EXPIRE):
"""Utility function to acquire a lock before processing the request"""
lock = cache.cache.get(name)
if lock:
acquire_lock.lock = lock
return False
return cache.cache.add(name, value, timeout)
acquire_lock.lock = None
def release_lock(name):
"""Utility function to release a lock"""
return cache.cache.delete(name)
def wait_for(lock_name, value, wait=10, timeout=LOCK_EXPIRE):
"""Utility function to wait until the given lock has been released"""
old_lock = None
if not acquire_lock(lock_name, value, timeout):
logger.warn(
......@@ -112,10 +126,19 @@ def wait_for(lock_name, value, wait=10, timeout=LOCK_EXPIRE):
@celery.task
def ping_backend():
if bui.standalone:
logger.debug(bui.cli.status())
bui.client.status()
else:
for server, backend in iteritems(bui.cli.servers):
logger.debug(bui.cli.status(agent=server))
def __status(server):
(serv, back) = server
try:
return bui.client.status(agent=serv)
except BUIserverException:
return False
map(
__status,
iteritems(bui.client.servers)
)
@celery.task(bind=True)
......@@ -127,7 +150,7 @@ def backup_running(self):
try:
cache.cache.set(
'backup_running_result',
bui.cli.is_one_backup_running(),
bui.client.is_one_backup_running(),
120
)
finally:
......@@ -143,18 +166,46 @@ def get_all_backups(self):
try:
backups = {}
if bui.standalone:
for cli in bui.cli.get_all_clients():
backups[cli['name']] = bui.cli.get_client(cli['name'])
for cli in bui.client.get_all_clients():
backups[cli['name']] = bui.client.get_client(cli['name'])
else:
for serv in bui.cli.servers:
for serv in bui.client.servers:
backups[serv] = {}
for cli in bui.cli.get_all_clients(agent=serv):
backups[serv][cli['name']] = bui.cli.get_client(cli['name'], agent=serv)
for cli in bui.client.get_all_clients(agent=serv):
backups[serv][cli['name']] = bui.client.get_client(cli['name'], agent=serv)
cache.cache.set('all_backups', backups, 3600)
finally:
release_lock(self.name)
@celery.task(bind=True)
def get_all_clients_reports(self):
# run once at the time, if one task was already running, we just discard
# the new attempt
if not acquire_lock(self.name):
return None
try:
reports = {}
if bui.standalone:
reports = bui.client.get_clients_report(bui.client.get_all_clients())
else:
for serv in bui.client.servers:
reports[serv] = bui.client.get_clients_report(bui.client.get_all_clients(agent=serv), serv)
cache.cache.set('all_clients_reports', reports, 3600)
finally:
release_lock(self.name)
@celery.task
def cleanup_expired_sessions():
def expires(sess):
ret = session_manager.invalidate_session_by_id(sess.uuid)
if ret:
session_manager.delete_session_by_id(sess.uuid)
return ret
map(expires, session_manager.get_expired_sessions())
@celery.task
def cleanup_restore():
tasks = Task.query.filter_by(task='perform_restore').all()
......@@ -177,7 +228,7 @@ def cleanup_restore():
path = task.result.get('path')
if path:
if server:
if not bui.cli.del_file(path, agent=server):
if not bui.client.del_file(path, agent=server):
logger.warn("'{}' already removed".format(path))
else:
if os.path.isfile(path):
......@@ -214,7 +265,7 @@ def perform_restore(self, client, backup,
fmt)
self.update_state(state='STARTED', meta={'step': 'doing'})
archive, err = bui.cli.restore_files(
archive, err = bui.client.restore_files(
client,
backup,
files,
......@@ -224,14 +275,9 @@ def perform_restore(self, client, backup,
server
)
if not archive:
if err:
self.update_state(state='FAILURE', meta={'error': err})
else:
if not err:
err = 'Something went wrong while restoring'
self.update_state(
state='FAILURE',
meta={'error': err}
)
self.update_state(state='FAILURE', meta={'error': err})
logger.error('FAILURE: {}'.format(err))
else:
ret = {
......@@ -253,11 +299,18 @@ def perform_restore(self, client, backup,
if err:
# make the task crash
raise BUIserverException(err)
raise Exception(err)
return ret
def force_scheduling_now():
"""Force scheduling some tasks now"""
get_all_backups.delay()
backup_running.delay()
get_all_clients_reports.delay()
@ns.route('/status/<task_id>', endpoint='async_restore_status')
@ns.doc(
params={
......@@ -287,7 +340,7 @@ class AsyncRestoreStatus(Resource):
db.session.commit()
task.revoke()
err = str(task.result)
self.abort(500, err)
self.abort(502, err)
if task.state == 'SUCCESS':
if not task.result:
self.abort(500, 'The task did not return anything')
......@@ -379,13 +432,13 @@ class AsyncGetFile(Resource):
mimetype='application/zip')
resp.set_cookie('fileDownload', 'true')
except Exception as e:
bui.cli.logger.error(str(e))
bui.client.logger.error(str(e))
self.abort(500, str(e))
return resp
def stream_file(self, path, filename, server):
socket = bui.cli.get_file(path, server)
socket = bui.client.get_file(path, server)
if not socket:
self.abort(500)
lengthbuf = socket.recv(8)
......@@ -723,6 +776,106 @@ class AsyncHistory(History):
# redirect to synchronous API call
# FIXME: Since we subclass the original code, we don't need the
# redirect anymore if the redirection is problematic
return redirect(url_for('api.history', client=client, server=server, start=args['start'], end=args['end']))
return redirect(
url_for(
'api.history',
client=client,
server=server,
start=args['start'],
end=args['end']
)
)
return self._get_backup_history(client, server, res)
@ns.route('/report',
'/<server>/report',
endpoint='async_clients_report')
@ns.doc(
params={
'server': 'Which server to collect data from when in multi-agent mode',
},
)
class AsyncClientsReport(ClientsReport):
"""The :class:`burpui.api.async.AsyncClientsReport` resource allows you to
access general reports about your clients.
This resource is part of the :mod:`burpui.api.clients` module.
An optional ``GET`` parameter called ``serverName`` is supported when
running in multi-agent mode.
"""
@api.cache.cached(timeout=1800, key_prefix=cache_key)
@ns.marshal_with(
ClientsReport.report_fields,
code=200,
description='Success',
strict=False
)
@ns.expect(ClientsReport.parser)
@ns.doc(
responses={
403: 'Insufficient permissions',
500: 'Internal failure',
},
)
def get(self, server=None):
"""Returns a global report about all the clients of a given server
**GET** method provided by the webservice.
The *JSON* returned is:
::
{
"backups": [
{
"name": "client1",
"number": 15
},
{
"name": "client2",
"number": 1
}
],
"clients": [
{
"name": "client1",
"stats": {
"total": 296377,
"totsize": 57055793698,
"windows": "unknown"
}
},
{
"name": "client2",
"stats": {
"total": 3117,
"totsize": 5345361,
"windows": "true"
}
}
]
}
The output is filtered by the :mod:`burpui.misc.acl` module so that you
only see stats about the clients you are authorized to.
:param server: Which server to collect data from when in multi-agent
mode
:type server: str
:returns: The *JSON* described above
"""
server = server or self.parser.parse_args()['serverName']
self._check_acl(server)
res = cache.cache.get('all_clients_reports')
if res is None:
# redirect to synchronous API call
# FIXME: Since we subclass the original code, we don't need the
# redirect anymore if the redirection is problematic
return redirect(url_for('api.clients_report', server=server))
return self._get_clients_reports(res, server)
......@@ -70,7 +70,7 @@ class ServerBackup(Resource):
self.is_admin)):
self.abort(403, 'You are not allowed to access this client')
try:
return {'is_server_backup': bui.cli.is_server_backup(name, server)}
return {'is_server_backup': bui.client.is_server_backup(name, server)}