Skip to content
Commits on Source (122)
Hide whitespace changes
Inline Side-by-side
.gitignore
View file @ d19f6460
*.pyc
*.swp
*.mo
burpui-dev.cfg*
burpui/RELEASE
devel.sh
*.egg*
.coverage
.coveragerc
dist
_build
.tags
......
CHANGELOG.rst
View file @ d19f6460
Changelog
=========
0.4.0 (11/23/2016)
------------------
- **BREAKING**: The database schema evolved. In order to apply these modifications, you **MUST** run the ``bui-manage db upgrade`` command after upgrading
- **BREAKING**: Plain text passwords are deprecated since v0.3.0 and are now disabled by default
- **BREAKING**: The default *version* setting has been set to ``2`` instead of ``1``
- Add: new `bui-manage setup_burp <https://git.ziirish.me/ziirish/burp-ui/merge_requests/40#note_1767>`_ command
- Add: new `docker image <https://git.ziirish.me/ziirish/burp-ui/merge_requests/40#note_1763>`_
- Add: manage `user sessions <https://git.ziirish.me/ziirish/burp-ui/merge_requests/6>`_
- Add: `French translation <https://git.ziirish.me/ziirish/burp-ui/merge_requests/4>`_
- Fix: issue `#151 <https://git.ziirish.me/ziirish/burp-ui/issues/151>`_
- Fix: issue `#154 <https://git.ziirish.me/ziirish/burp-ui/issues/154>`_
- Fix: issue `#158 <https://git.ziirish.me/ziirish/burp-ui/issues/158>`_
- Fix: issue `#163 <https://git.ziirish.me/ziirish/burp-ui/issues/163>`_
- Fix: issue `#164 <https://git.ziirish.me/ziirish/burp-ui/issues/164>`_
- Fix: issue `#166 <https://git.ziirish.me/ziirish/burp-ui/issues/166>`_
- Fix: issue `#169 <https://git.ziirish.me/ziirish/burp-ui/issues/169>`_
- Fix: issue `#171 <https://git.ziirish.me/ziirish/burp-ui/issues/171>`_
- Fix: issue `#172 <https://git.ziirish.me/ziirish/burp-ui/issues/172>`_
- Fix: issue `#173 <https://git.ziirish.me/ziirish/burp-ui/issues/173>`_
- Fix: issue `#174 <https://git.ziirish.me/ziirish/burp-ui/issues/174>`_
- Various bugfix
- `Full changelog <https://git.ziirish.me/ziirish/burp-ui/compare/v0.3.0...v0.4.0>`__
0.3.0 (08/15/2016)
------------------
......
Dockerfile
View file @ d19f6460
......@@ -2,7 +2,7 @@ FROM debian:jessie
MAINTAINER hi+burpui@ziirish.me
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y supervisor logrotate locales wget curl python2.7-dev git python-virtualenv gunicorn python-pip cron libffi-dev redis-server \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y supervisor logrotate locales wget python2.7-dev git gunicorn python-pip cron libffi-dev netcat \
&& update-locale LANG=C.UTF-8 LC_MESSAGES=POSIX \
&& locale-gen en_US.UTF-8 \
&& dpkg-reconfigure -f noninteractive locales \
......@@ -21,8 +21,9 @@ RUN chmod 755 /app/setup/install
RUN /app/setup/install
EXPOSE 5000/tcp
EXPOSE 4971/tcp
EXPOSE 4972/tcp
VOLUME ["/var/spool/burp"]
VOLUME ["/etc/burp"]
ENTRYPOINT ["/app/init"]
CMD ["app:start"]
README.rst
View file @ d19f6460
Badges
======
.. image:: https://git.ziirish.me/ci/projects/1/status.png?ref=master
:target: https://git.ziirish.me/ci/projects/1?ref=master
.. image:: https://git.ziirish.me/ci/projects/1/status.png?ref=stable
:target: https://git.ziirish.me/ziirish/burp-ui/pipelines
:alt: Build Status
.. image:: https://readthedocs.org/projects/burp-ui/badge/?version=latest
:target: https://readthedocs.org/projects/burp-ui/?badge=latest
.. image:: https://git.ziirish.me/ziirish/burp-ui/badges/stable/coverage.svg
:target: https://git.ziirish.me/ziirish/burp-ui/pipelines
:alt: Test coverage
.. image:: https://readthedocs.org/projects/burp-ui/badge/?version=stable
:target: https://readthedocs.org/projects/burp-ui/?badge=stable
:alt: Documentation Status
Introduction
......@@ -15,8 +19,8 @@ Introduction
Screenshots
-----------
.. image:: https://git.ziirish.me/ziirish/burp-ui/raw/master/docs/_static/burp-ui.gif
:target: https://git.ziirish.me/ziirish/burp-ui/blob/master/docs/_static/burp-ui.gif
.. image:: https://git.ziirish.me/ziirish/burp-ui/raw/stable/docs/_static/burp-ui.gif
:target: https://git.ziirish.me/ziirish/burp-ui/blob/stable/docs/_static/burp-ui.gif
Demo
......@@ -59,13 +63,13 @@ Documentation
The documentation is hosted on `readthedocs <https://readthedocs.org>`_ at the
following address: `burp-ui.readthedocs.io
<https://burp-ui.readthedocs.io/en/latest/>`_
<https://burp-ui.readthedocs.io/en/stable/>`_
FAQ
===
A `FAQ <https://burp-ui.readthedocs.io/en/latest/faq.html>`_ is available with
A `FAQ <https://burp-ui.readthedocs.io/en/stable/faq.html>`_ is available with
the documentation.
......@@ -73,7 +77,7 @@ Community
=========
Please refer to the `Contributing
<https://burp-ui.readthedocs.io/en/latest/contributing.html>`_ page.
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_ page.
Notes
......@@ -81,9 +85,11 @@ Notes
Feel free to report any issues on my `gitlab
<https://git.ziirish.me/ziirish/burp-ui/issues>`_.
I have closed the *github tracker* to have a unique tracker system.
Also please, read the `Contributing
<https://burp-ui.readthedocs.io/en/latest/contributing.html>`_
<https://burp-ui.readthedocs.io/en/stable/contributing.html>`_
page before reporting any issue to make sure we have all the informations to
help you.
......@@ -93,7 +99,7 @@ Licenses
``Burp-UI`` is released under the BSD 3-clause `License`_.
But this project is built on top of other tools listed here:
But this project is built on top of other tools. Here is a non exhaustive list:
- `d3.js <http://d3js.org/>`_
- `nvd3.js <http://nvd3.org/>`_
......@@ -109,7 +115,7 @@ But this project is built on top of other tools listed here:
- `AngularStrap <http://mgcrea.github.io/angular-strap/>`_
- `lodash <https://github.com/lodash/lodash>`_
- `DataTables <http://datatables.net/>`_
- Home-made `favicon <https://git.ziirish.me/ziirish/burp-ui/blob/master/burpui/static/images/favicon.ico>`_ based on pictures from `simpsoncrazy <http://www.simpsoncrazy.com/pictures/homer>`_
- Home-made `favicon <https://git.ziirish.me/ziirish/burp-ui/blob/stable/burpui/static/images/favicon.ico>`_ based on pictures from `simpsoncrazy <http://www.simpsoncrazy.com/pictures/homer>`_
Also note that this project is made with the Awesome `Flask`_ micro-framework.
......@@ -120,13 +126,13 @@ Thanks
Thank you all for your feedbacks and bug reports. Those are making the project
moving forward.
Thank you to the `Flask`_'s developers and community.
Thank you to the `Flask`_ developers and community.
Special Thanks to Graham Keeling for its great piece of software! This project
would not exist without `Burp`_.
.. _Flask: http://flask.pocoo.org/
.. _License: https://git.ziirish.me/ziirish/burp-ui/blob/master/LICENSE
.. _License: https://git.ziirish.me/ziirish/burp-ui/blob/stable/LICENSE
.. _Burp: http://burp.grke.org/
.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/master/share/burpui/etc/burpui.sample.cfg
.. _burpui.cfg: https://git.ziirish.me/ziirish/burp-ui/blob/stable/share/burpui/etc/burpui.sample.cfg
babel.cfg 0 → 100644
View file @ d19f6460
[python: **.py]
encoding = utf-8
[jinja2: **/templates/**.html]
encoding = utf-8
extensions=jinja2.ext.autoescape,jinja2.ext.with_
[jinja2: **/templates/**.js]
encoding = utf-8
extensions=jinja2.ext.autoescape,jinja2.ext.with_
bui-agent
View file @ d19f6460
#!/bin/bash
python ./burpui -m agent $@
python ./burpui -m agent "$@"
bui-celery
View file @ d19f6460
#!/bin/bash
python ./burpui -m celery $@
python ./burpui -m celery "$@"
bui-manage
View file @ d19f6460
#!/bin/bash
python ./burpui -m manage $@
python ./burpui -m manage "$@"
burp-ui
View file @ d19f6460
#!/bin/bash
python ./burpui $@
python ./burpui "$@"
burpui/__init__.py
View file @ d19f6460
......@@ -9,437 +9,17 @@ jQuery/Bootstrap
.. moduleauthor:: Ziirish <hi+burpui@ziirish.me>
"""
import os
import re
import sys
import logging
import warnings
from logging import Formatter
if sys.version_info < (3, 0):
reload(sys)
sys.setdefaultencoding('utf-8')
else:
basestring = str
__title__ = 'burp-ui'
__author__ = 'Benjamin SANS (Ziirish)'
__author_email__ = 'hi+burpui@ziirish.me'
__url__ = 'https://git.ziirish.me/ziirish/burp-ui'
__doc__ = 'https://burp-ui.readthedocs.io/en/latest/'
__description__ = ('Burp-UI is a web-ui for burp backup written in python with '
'Flask and jQuery/Bootstrap')
__license__ = 'BSD 3-clause'
__version__ = open(
os.path.join(os.path.dirname(os.path.realpath(__file__)), 'VERSION')
).read().rstrip()
try: # pragma: no cover
__release__ = open(
os.path.join(os.path.dirname(os.path.realpath(__file__)), 'RELEASE')
).read().rstrip()
except: # pragma: no cover
__release__ = 'unknown'
from .app import init
warnings.simplefilter('always', RuntimeWarning)
def parse_db_setting(string):
parts = re.search(
'(?:(?P<backend>\w+)(?:\+(?P<driver>\w+))?://)?'
'(?:(?P<user>\w+)(?::?(?P<pass>.+))?@)?'
'(?P<host>[\w_.-]+):?(?P<port>\d+)?(?:/(?P<db>\w+))?',
string
)
if not parts:
raise ValueError('Unable to parse the db: "{}"'.format(string))
back = parts.group('backend') or ''
user = parts.group('user') or None
pwd = parts.group('pass') or None
host = parts.group('host') or ''
port = parts.group('port') or ''
db = parts.group('db') or ''
return (back, user, pwd, host, port, db)
def get_redis_server(myapp):
host = 'localhost'
port = 6379
if myapp.redis and myapp.redis.lower() != 'none':
try:
back, user, pwd, host, port, db = parse_db_setting(myapp.redis)
host = host or 'localhost'
try:
port = int(port)
except (ValueError, IndexError):
port = 6379
except ValueError:
pass
return host, port, pwd
def create_db(myapp):
"""Create the SQLAlchemy instance if possible
:param myapp: Application context
:type myapp: :class:`burpui.server.BUIServer`
"""
if myapp.config['WITH_SQL']:
from .ext.sql import db
myapp.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db.init_app(myapp)
return db
return None
def create_celery(myapp, warn=True):
"""Create the Celery app if possible
:param myapp: Application context
:type myapp: :class:`burpui.server.BUIServer`
"""
if myapp.config['WITH_CELERY']:
from .ext.async import celery
host, oport, pwd = get_redis_server(myapp)
odb = 2
if isinstance(myapp.use_celery, basestring):
try:
(_, _, pwd, host, port, db) = parse_db_setting(myapp.use_celery)
if not port:
port = oport
if not db:
db = odb
else:
try:
db = int(db)
except ValueError:
db = odb
except ValueError:
pass
else:
db = odb
port = oport
if pwd:
redis_url = 'redis://:{}@{}:{}/{}'.format(pwd, host, port, db)
else:
redis_url = 'redis://{}:{}/{}'.format(host, port, db)
myapp.config['CELERY_BROKER_URL'] = myapp.config['BROKER_URL'] = \
redis_url
myapp.config['CELERY_RESULT_BACKEND'] = redis_url
celery.conf.update(myapp.config)
if not hasattr(celery, 'flask_app'):
celery.flask_app = myapp
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with myapp.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
return celery
if warn:
message = 'Something went wrong while initializing celery worker.\n' \
'Maybe it is not enabled in your conf ' \
'({}).'.format(myapp.config['CFG'])
warnings.warn(
message,
RuntimeWarning
)
return None
def init(conf=None, verbose=0, logfile=None, gunicorn=True, unittest=False, debug=False):
"""Initialize the whole application.
:param conf: Configuration file to use
:type conf: str
:param verbose: Set the verbosity level
:type verbose: int
:param logfile: Store the logs in the given file
:type logfile: str
:param gunicorn: Enable gunicorn engine instead of flask's default
:type gunicorn: bool
:param unittest: Are we running tests (used for test only)
:type unittest: bool
:param debug: Enable debug mode
:type debug: bool
:returns: A :class:`burpui.server.BUIServer` object
"""
from flask_login import LoginManager
from flask_bower import Bower
from .utils import basic_login_from_request, ReverseProxied, lookup_file
from .server import BUIServer as BurpUI
from .routes import view
from .api import api, apibp
from .ext.cache import cache
logger = logging.getLogger('burp-ui')
# The debug argument used to be a boolean so we keep supporting this format
if isinstance(verbose, bool):
if verbose:
verbose = logging.DEBUG
else:
verbose = logging.CRITICAL
else:
levels = [
logging.CRITICAL,
logging.ERROR,
logging.WARNING,
logging.INFO,
logging.DEBUG
]
if verbose >= len(levels):
verbose = len(levels) - 1
if not verbose:
verbose = 0
verbose = levels[verbose]
if logfile:
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
logfile,
maxBytes=1024 * 1024 * 100,
backupCount=5
)
else:
from logging import StreamHandler
handler = StreamHandler()
if verbose > logging.DEBUG:
LOG_FORMAT = (
'[%(asctime)s] %(levelname)s in '
'%(module)s.%(funcName)s: %(message)s'
)
else:
LOG_FORMAT = (
'-' * 80 + '\n' +
'%(levelname)s in %(module)s.%(funcName)s ' +
'[%(pathname)s:%(lineno)d]:\n' +
'%(message)s\n' +
'-' * 80
)
handler.setLevel(verbose)
handler.setFormatter(Formatter(LOG_FORMAT))
logger.setLevel(verbose)
logger.addHandler(handler)
logger.debug(
'conf: {}\n'.format(conf) +
'verbose: {}\n'.format(logging.getLevelName(verbose)) +
'logfile: {}\n'.format(logfile) +
'gunicorn: {}\n'.format(gunicorn) +
'debug: {}\n'.format(debug) +
'unittest: {}'.format(unittest)
)
if not unittest:
from ._compat import patch_json
patch_json()
# We initialize the core
app = BurpUI()
if verbose:
app.enable_logger()
app.gunicorn = gunicorn
app.config['CFG'] = None
# FIXME: strange behavior when bundling errors
# app.config['BUNDLE_ERRORS'] = True
app.config['REMEMBER_COOKIE_HTTPONLY'] = True
if debug and not gunicorn: # pragma: no cover
app.config['DEBUG'] = True and not unittest
app.config['TESTING'] = True and not unittest
# Still need to test conf file here because the init function can be called
# by gunicorn directly
if conf:
app.config['CFG'] = lookup_file(conf, guess=False)
else:
app.config['CFG'] = lookup_file()
logger.info('Using configuration: {}'.format(app.config['CFG']))
app.setup(app.config['CFG'])
if debug:
app.config.setdefault('TEMPLATES_AUTO_RELOAD', True)
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.jinja_env.globals.update(
isinstance=isinstance,
list=list,
version_id='{}-{}'.format(__version__, __release__),
config=app.config
)
# manage application secret key
if app.secret_key and (app.secret_key.lower() == 'none' or
(app.secret_key.lower() == 'random' and gunicorn)):
logger.warning('Your setup is not secure! Please consider setting a'
' secret key in your configuration file')
app.secret_key = 'Burp-UI'
if not app.secret_key or app.secret_key.lower() == 'random':
from base64 import b64encode
app.secret_key = b64encode(os.urandom(256))
app.wsgi_app = ReverseProxied(app.wsgi_app, app)
# Manage gunicorn special tricks & improvements
if gunicorn: # pragma: no cover
logger.info('Using gunicorn')
from werkzeug.contrib.fixers import ProxyFix
app.wsgi_app = ProxyFix(app.wsgi_app)
if app.storage and app.storage.lower() != 'default':
try:
# Session setup
if not app.session_db or app.session_db.lower() != 'none':
from redis import Redis
from flask_session import Session
host, port, pwd = get_redis_server(app)
db = 0
if app.session_db and app.session_db.lower() != 'default':
try:
(_, _, pwd, host, port, db) = \
parse_db_setting(app.session_db)
except ValueError as exp:
logger.warning(str(exp))
try:
db = int(db)
except ValueError:
db = 0
logger.debug('Using redis://guest:****@{}:{}/{}'.format(
host,
port,
db)
)
red = Redis(host=host, port=port, db=db, password=pwd)
app.config['SESSION_TYPE'] = 'redis'
app.config['SESSION_REDIS'] = red
app.config['SESSION_USE_SIGNER'] = app.secret_key is not None
app.config['SESSION_PERMANENT'] = False
ses = Session()
ses.init_app(app)
# Cache setup
if not app.cache_db or app.cache_db.lower() != 'none':
host, port, pwd = get_redis_server(app)
db = 1
if app.cache_db and app.cache_db.lower() != 'default':
try:
(_, _, pwd, host, port, db) = \
parse_db_setting(app.cache_db)
except ValueError as exp:
logger.warning(str(exp))
try:
db = int(db)
except ValueError:
db = 1
logger.debug('Using redis://guest:****@{}:{}/{}'.format(
host,
port,
db)
)
cache.init_app(
app,
config={
'CACHE_TYPE': 'redis',
'CACHE_REDIS_HOST': host,
'CACHE_REDIS_PORT': port,
'CACHE_REDIS_PASSWORD': pwd,
'CACHE_REDIS_DB': db
}
)
# clear cache at startup in case we removed or added servers
with app.app_context():
cache.clear()
else:
cache.init_app(app)
except Exception as e:
logger.warning('Unable to initialize redis: {}'.format(str(e)))
cache.init_app(app)
else:
cache.init_app(app)
# Create celery app if enabled
create_celery(app, warn=False)
# Create SQLAlchemy if enabled
create_db(app)
# We initialize the API
api.version = __version__
api.release = __release__
api.__url__ = __url__
api.__doc__ = __doc__
api.load_all()
app.register_blueprint(apibp)
# Then we load our routes
view.__url__ = __url__
view.__doc__ = __doc__
app.register_blueprint(view)
# And the login_manager
app.login_manager = LoginManager()
app.login_manager.login_view = 'view.login'
app.login_manager.login_message_category = 'info'
app.login_manager.session_protection = 'strong'
app.login_manager.init_app(app)
app.config.setdefault(
'BOWER_COMPONENTS_ROOT',
os.path.join('static', 'vendor')
)
app.config.setdefault('BOWER_REPLACE_URL_FOR', True)
bower = Bower()
bower.init_app(app)
@app.before_request
def setup_request():
# make sure to store secure cookie if required
if app.scookie:
from flask import request
criteria = [
request.is_secure,
request.headers.get('X-Forwarded-Proto', 'http') == 'https'
]
app.config['SESSION_COOKIE_SECURE'] = \
app.config['REMEMBER_COOKIE_SECURE'] = any(criteria)
@app.login_manager.user_loader
def load_user(userid):
"""User loader callback"""
if app.auth != 'none':
return app.uhandler.user(userid)
return None
@app.login_manager.request_loader
def load_user_from_request(request):
"""User loader from request callback"""
if app.auth != 'none':
return basic_login_from_request(request, app)
return app
if sys.version_info < (3, 0): # pragma: no cover
reload(sys)
sys.setdefaultencoding('utf-8')
# backward compatibility
create_app = init
burpui/__main__.py
View file @ d19f6460
......@@ -27,7 +27,7 @@ def parse_args(mode=True, name=None):
parser.add_argument('-v', '--verbose', dest='log', help='increase output verbosity (e.g., -vv is more verbose than -v)', action='count')
parser.add_argument('-d', '--debug', dest='debug', help='enable debug mode', action='store_true') # alias for -v
parser.add_argument('-V', '--version', dest='version', help='print version and exit', action='store_true')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-l', '--logfile', dest='logfile', help='output logs in defined file', metavar='<FILE>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('remaining', nargs=REMAINDER)
......@@ -39,7 +39,7 @@ def parse_args(mode=True, name=None):
options = parser.parse_args()
if options.version:
from burpui import __title__, __version__, __release__
from burpui.app import __title__, __version__, __release__
ver = '{}: v{}'.format(name or __title__, __version__)
if options.log:
ver = '{} ({})'.format(ver, __release__)
......@@ -113,16 +113,20 @@ def celery():
from burpui.utils import lookup_file
parser = ArgumentParser('bui-celery')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
conf = lookup_file()
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
# make conf path absolute
......@@ -132,7 +136,6 @@ def celery():
os.chdir(ROOT)
env = os.environ
env['BUI_CONFIG'] = conf
args = [
......@@ -151,17 +154,21 @@ def manage():
from burpui.utils import lookup_file
parser = ArgumentParser('bui-manage')
parser.add_argument('-c', '--config', dest='config', help='configuration file', metavar='<CONFIG>')
parser.add_argument('-c', '--config', dest='config', help='burp-ui configuration file', metavar='<CONFIG>')
parser.add_argument('-i', '--migrations', dest='migrations', help='migrations directory', metavar='<MIGRATIONSDIR>')
parser.add_argument('-m', '--mode', dest='mode', help='application mode', metavar='<agent|server|worker|manage>')
parser.add_argument('remaining', nargs=REMAINDER)
options, unknown = parser.parse_known_args()
env = os.environ
if options.config:
conf = lookup_file(options.config, guess=False)
else:
conf = lookup_file()
if 'BUI_CONFIG' in env:
conf = env['BUI_CONFIG']
else:
conf = lookup_file()
check_config(conf)
if options.migrations:
......@@ -169,14 +176,16 @@ def manage():
else:
migrations = lookup_file('migrations', directory=True)
env = os.environ
env['BUI_CONFIG'] = conf
if migrations:
env['BUI_MIGRATIONS'] = migrations
if os.path.isdir('burpui'):
env['FLASK_APP'] = 'burpui/cli.py'
else:
env['FLASK_APP'] = 'burpui.cli'
args = [
sys.executable,
os.path.join(ROOT, 'manage.py'),
'flask'
]
args += unknown
args += [x for x in options.remaining if x != '--']
......
burpui/_rtfd.py
View file @ d19f6460
......@@ -12,4 +12,6 @@
from . import create_app
app = create_app(conf='/dev/null', gunicorn=False)
# This is a lie we are not really unittesting, but we want to avoid the v2
# errors
app = create_app(conf='/dev/null', gunicorn=False, unittest=True)
burpui/agent.py
View file @ d19f6460
......@@ -30,7 +30,7 @@ from .config import config
G_PORT = 10000
G_BIND = u'::'
G_SSL = False
G_VERSION = 1
G_VERSION = 2
G_SSLCERT = u''
G_SSLKEY = u''
G_PASSWORD = u'password'
......@@ -48,7 +48,7 @@ class BurpHandler(BUIbackend):
foreign = BUIbackend.__abstractmethods__
BUIbackend.__abstractmethods__ = frozenset()
def __init__(self, vers=1, logger=None, conf=None):
def __init__(self, vers=2, logger=None, conf=None):
self.vers = vers
self.logger = logger
......@@ -95,6 +95,7 @@ class BUIAgent(BUIbackend, BUIlogging):
def __init__(self, conf=None, level=0, logfile=None, debug=False):
self.debug = debug
self.padding = 1
level = level or 0
if level > logging.NOTSET:
logging.addLevelName(DISCLOSURE, 'DISCLOSURE')
levels = [
......@@ -141,8 +142,9 @@ class BUIAgent(BUIbackend, BUIlogging):
self.sslcert = self.conf.safe_get('sslcert')
self.sslkey = self.conf.safe_get('sslkey')
self.password = self.conf.safe_get('password')
self.conf.setdefault('BUI_AGENT', True)
self.cli = BurpHandler(self.vers, self.logger, self.conf)
self.client = BurpHandler(self.vers, self.logger, self.conf)
pool = Pool(10000)
if not self.ssl:
self.server = StreamServer((self.bind, self.port), self.handle, spawn=pool)
......@@ -176,13 +178,13 @@ class BUIAgent(BUIbackend, BUIlogging):
return
try:
if j['func'] == 'proxy_parser':
parser = self.cli.get_parser()
parser = self.client.get_parser()
if j['args']:
res = json.dumps(getattr(parser, j['method'])(**j['args']))
else:
res = json.dumps(getattr(parser, j['method'])())
elif j['func'] == 'restore_files':
res, err = getattr(self.cli, j['func'])(**j['args'])
res, err = getattr(self.client, j['func'])(**j['args'])
if err:
self.request.sendall(b'ER')
self.request.sendall(struct.pack('!Q', len(err)))
......@@ -195,7 +197,7 @@ class BUIAgent(BUIbackend, BUIlogging):
err = None
if not path.startswith('/'):
err = 'The path must be absolute! ({})'.format(path)
if not path.startswith(self.cli.tmpdir):
if not path.startswith(self.client.tmpdir):
err = 'You are not allowed to access this path: ' \
'({})'.format(path)
if err:
......@@ -226,7 +228,7 @@ class BUIAgent(BUIbackend, BUIlogging):
err = None
if not path.startswith('/'):
err = 'The path must be absolute! ({})'.format(path)
if not path.startswith(self.cli.tmpdir):
if not path.startswith(self.client.tmpdir):
err = 'You are not allowed to access this path: ' \
'({})'.format(path)
if err:
......@@ -246,26 +248,24 @@ class BUIAgent(BUIbackend, BUIlogging):
import hmac
import hashlib
from base64 import b64decode
pickles = j['args']
pickles = j['args'].encode(encoding='utf-8')
key = u'{}{}'.format(self.password, j['func'])
key = key.encode(encoding='utf-8')
bytes_pickles = pickles.encode(encoding='utf-8')
bytes_pickles = pickles
digest = hmac.new(key, bytes_pickles, hashlib.sha1).hexdigest()
if digest != j['digest']:
raise BUIserverException('Integrity check failed')
raise BUIserverException('Integrity check failed: {} != {}'.format(digest, j['digest']))
j['args'] = pickle.loads(b64decode(pickles))
res = json.dumps(getattr(self.cli, j['func'])(**j['args']))
res = json.dumps(getattr(self.client, j['func'])(**j['args']))
else:
res = json.dumps(getattr(self.cli, j['func'])())
res = json.dumps(getattr(self.client, j['func'])())
self._logger('info', 'result: {}'.format(res))
self.request.sendall(b'OK')
# should not happen
except Exception as e:
raise BUIserverException(str(e))
except BUIserverException as e:
except (BUIserverException, Exception) as e:
self.request.sendall(b'ER')
res = str(e)
self._logger('error', 'Forwarding Exception: {}'.format(res))
self._logger('error', traceback.format_exc())
self._logger('warning', 'Forwarding Exception: {}'.format(res))
self.request.sendall(struct.pack('!Q', len(res)))
self.request.sendall(res.encode('UTF-8'))
return
......
burpui/api/__init__.py
View file @ d19f6460
......@@ -41,12 +41,12 @@ def api_login_required(func):
"""decorator"""
if request.method in EXEMPT_METHODS:
return func(*args, **kwargs)
try:
name = func.func_name
except: # pragma: no cover
name = func.__name__
# 'func' is a Flask.view.MethodView so we have access to some special
# params
cls = func.view_class
login_required = getattr(cls, 'login_required', True)
if (bui.auth != 'none' and
name not in api.LOGIN_NOT_REQUIRED and
login_required and
not bui.config.get('LOGIN_DISABLED', False)):
if not current_user.is_authenticated:
if request.headers.get('X-From-UI', False):
......@@ -68,7 +68,6 @@ class Api(ApiPlus):
release = None
__doc__ = None
__url__ = None
LOGIN_NOT_REQUIRED = []
CELERY_REQUIRED = ['async']
def load_all(self):
......@@ -110,6 +109,16 @@ class Api(ApiPlus):
return decorated
return decorator
def disabled_on_demo(self):
def decorator(func):
@wraps(func)
def decorated(resource, *args, **kwargs):
if config['BUI_DEMO']:
resource.abort(405, 'Sorry, this feature is not available on the demo')
return func(resource, *args, **kwargs)
return decorated
return decorator
def namespace(self, *args, **kwargs):
"""A namespace factory
......
burpui/api/admin.py
View file @ d19f6460
......@@ -9,6 +9,8 @@
"""
from . import api
from ..server import BUIServer # noqa
from ..sessions import session_manager
from ..utils import NOTIF_OK
from .custom import fields, Resource
# from ..exceptions import BUIserverException
......@@ -24,6 +26,16 @@ user_fields = ns.model('Users', {
'name': fields.String(required=True, description='User name'),
'backend': fields.String(required=True, description='Backend name'),
})
session_fields = ns.model('Sessions', {
'uuid': fields.String(description='Session id'),
'ip': fields.String(description='IP address'),
'ua': fields.String(description='User-Agent'),
'permanent': fields.Boolean(description='Remember cookie'),
'api': fields.Boolean(description='API login'),
'expire': fields.DateTime(description='Expiration date'),
'timestamp': fields.DateTime(description='Last seen'),
'current': fields.Boolean(description='Is current session', default=False)
})
@ns.route('/me', endpoint='admin_me')
......@@ -68,6 +80,7 @@ class AuthUsers(Resource):
parser_mod = ns.parser()
parser_mod.add_argument('password', required=True, help='Password', location=('values', 'json'))
parser_mod.add_argument('backend', required=True, help='Backend', location=('values', 'json'))
parser_mod.add_argument('old_password', required=False, help='Old password', location=('values', 'json'))
parser_del = ns.parser()
parser_del.add_argument('backend', required=True, help='Backend', location='values')
......@@ -118,6 +131,7 @@ class AuthUsers(Resource):
})
return ret
@api.disabled_on_demo()
@api.acl_admin_required(message="Not allowed to create users")
@ns.expect(parser_add)
@ns.doc(
......@@ -159,6 +173,7 @@ class AuthUsers(Resource):
status = 201 if success else 200
return [[code, message]], status
@api.disabled_on_demo()
@api.acl_admin_required(message="Not allowed to delete this user")
@ns.expect(parser_del)
@ns.doc(
......@@ -199,6 +214,7 @@ class AuthUsers(Resource):
status = 201 if success else 200
return [[code, message]], status
@api.disabled_on_demo()
@api.acl_own_or_admin(key='name', message="Not allowed to modify this user")
@ns.expect(parser_mod)
@ns.doc(
......@@ -215,6 +231,9 @@ class AuthUsers(Resource):
"""Change user password"""
args = self.parser_mod.parse_args()
if not self.is_admin and not args['old_password']:
self.abort(400, "Old password required")
try:
handler = getattr(bui, 'uhandler')
except AttributeError:
......@@ -235,7 +254,8 @@ class AuthUsers(Resource):
success, message, code = backend.change_password(
name,
args['password']
args['password'],
args.get('old_password')
)
status = 201 if success else 200
return [[code, message]], status
......@@ -288,3 +308,68 @@ class AuthBackends(Resource):
})
return ret
@ns.route('/me/session',
'/me/session/<id>',
endpoint='user_sessions')
@ns.doc(
params={
'id': 'Session id',
}
)
class MySessions(Resource):
"""The :class:`burpui.api.admin.MySessions` resource allows you to
retrieve a list of sessions and invalidate them for the current user.
This resource is part of the :mod:`burpui.api.admin` module.
"""
@ns.marshal_list_with(session_fields, code=200, description='Success')
@ns.doc(
responses={
403: 'Insufficient permissions',
404: 'User not found',
},
)
def get(self, id=None):
"""Returns a list of sessions
**GET** method provided by the webservice.
:returns: Sessions
"""
if id:
return session_manager.get_session_by_id(id)
user = getattr(current_user, 'name', None)
if not user:
self.abort(404, 'User not found')
return session_manager.get_user_sessions(user)
@api.disabled_on_demo()
@ns.doc(
responses={
201: 'Success',
403: 'Insufficient permissions',
404: 'User or session not found',
400: 'Wrong request'
}
)
def delete(self, id=None):
"""Delete a given session
Note: ``id`` is mandatory
"""
if not id:
self.abort(400, 'Missing id')
user = getattr(current_user, 'name', None)
if not user:
self.abort(404, 'User not found')
store = session_manager.get_session_by_id(id)
if not store:
self.abort('Session not found')
if store.user != user:
self.abort(403, 'Insufficient permissions')
if session_manager.invalidate_session_by_id(store.uuid):
session_manager.delete_session_by_id(store.uuid)
return [NOTIF_OK, 'Session {} successfully revoked'.format(id)], 201
burpui/api/async.py
View file @ d19f6460
......@@ -12,14 +12,15 @@ import select
import struct
from . import api, cache_key
from .clients import RunningBackup
from .misc import History
from ..server import BUIServer # noqa
from .custom import Resource
from .clients import RunningBackup, ClientsReport
from ..exceptions import BUIserverException
from ..server import BUIServer # noqa
from ..sessions import session_manager
from ..ext.async import celery
from ..ext.cache import cache
from ..config import config
from ..exceptions import BUIserverException
from six import iteritems
from zlib import adler32
......@@ -52,9 +53,17 @@ BEAT_SCHEDULE = {
'task': '{}.backup_running'.format(ME),
'schedule': crontab(), # run every minute
},
'get-all-backups-bi-hourly': {
'get-all-backups-every-twenty-minutes': {
'task': '{}.get_all_backups'.format(ME),
'schedule': crontab(minute='23,53'), # every 30 minutes
'schedule': crontab(minute='*/20'), # every 20 minutes
},
'get-all-clients-reports-every-twenty-minutes': {
'task': '{}.get_all_clients_reports'.format(ME),
'schedule': crontab(minute='*/20'), # every 20 minutes
},
'cleanup-expired-sessions-daily': {
'task': '{}.cleanup_expired_sessions'.format(ME),
'schedule': crontab(hour='1'), # every day at 1
},
}
......@@ -76,19 +85,24 @@ else:
def acquire_lock(name, value='nyan', timeout=LOCK_EXPIRE):
"""Utility function to acquire a lock before processing the request"""
lock = cache.cache.get(name)
if lock:
acquire_lock.lock = lock
return False
return cache.cache.add(name, value, timeout)
acquire_lock.lock = None
def release_lock(name):
"""Utility function to release a lock"""
return cache.cache.delete(name)
def wait_for(lock_name, value, wait=10, timeout=LOCK_EXPIRE):
"""Utility function to wait until the given lock has been released"""
old_lock = None
if not acquire_lock(lock_name, value, timeout):
logger.warn(
......@@ -112,10 +126,19 @@ def wait_for(lock_name, value, wait=10, timeout=LOCK_EXPIRE):
@celery.task
def ping_backend():
if bui.standalone:
logger.debug(bui.cli.status())
bui.client.status()
else:
for server, backend in iteritems(bui.cli.servers):
logger.debug(bui.cli.status(agent=server))
def __status(server):
(serv, back) = server
try:
return bui.client.status(agent=serv)
except BUIserverException:
return False
map(
__status,
iteritems(bui.client.servers)
)
@celery.task(bind=True)
......@@ -127,7 +150,7 @@ def backup_running(self):
try:
cache.cache.set(
'backup_running_result',
bui.cli.is_one_backup_running(),
bui.client.is_one_backup_running(),
120
)
finally:
......@@ -143,18 +166,46 @@ def get_all_backups(self):
try:
backups = {}
if bui.standalone:
for cli in bui.cli.get_all_clients():
backups[cli['name']] = bui.cli.get_client(cli['name'])
for cli in bui.client.get_all_clients():
backups[cli['name']] = bui.client.get_client(cli['name'])
else:
for serv in bui.cli.servers:
for serv in bui.client.servers:
backups[serv] = {}
for cli in bui.cli.get_all_clients(agent=serv):
backups[serv][cli['name']] = bui.cli.get_client(cli['name'], agent=serv)
for cli in bui.client.get_all_clients(agent=serv):
backups[serv][cli['name']] = bui.client.get_client(cli['name'], agent=serv)
cache.cache.set('all_backups', backups, 3600)
finally:
release_lock(self.name)
@celery.task(bind=True)
def get_all_clients_reports(self):
# run once at the time, if one task was already running, we just discard
# the new attempt
if not acquire_lock(self.name):
return None
try:
reports = {}
if bui.standalone:
reports = bui.client.get_clients_report(bui.client.get_all_clients())
else:
for serv in bui.client.servers:
reports[serv] = bui.client.get_clients_report(bui.client.get_all_clients(agent=serv), serv)
cache.cache.set('all_clients_reports', reports, 3600)
finally:
release_lock(self.name)
@celery.task
def cleanup_expired_sessions():
def expires(sess):
ret = session_manager.invalidate_session_by_id(sess.uuid)
if ret:
session_manager.delete_session_by_id(sess.uuid)
return ret
map(expires, session_manager.get_expired_sessions())
@celery.task
def cleanup_restore():
tasks = Task.query.filter_by(task='perform_restore').all()
......@@ -177,7 +228,7 @@ def cleanup_restore():
path = task.result.get('path')
if path:
if server:
if not bui.cli.del_file(path, agent=server):
if not bui.client.del_file(path, agent=server):
logger.warn("'{}' already removed".format(path))
else:
if os.path.isfile(path):
......@@ -214,7 +265,7 @@ def perform_restore(self, client, backup,
fmt)
self.update_state(state='STARTED', meta={'step': 'doing'})
archive, err = bui.cli.restore_files(
archive, err = bui.client.restore_files(
client,
backup,
files,
......@@ -224,14 +275,9 @@ def perform_restore(self, client, backup,
server
)
if not archive:
if err:
self.update_state(state='FAILURE', meta={'error': err})
else:
if not err:
err = 'Something went wrong while restoring'
self.update_state(
state='FAILURE',
meta={'error': err}
)
self.update_state(state='FAILURE', meta={'error': err})
logger.error('FAILURE: {}'.format(err))
else:
ret = {
......@@ -253,11 +299,18 @@ def perform_restore(self, client, backup,
if err:
# make the task crash
raise BUIserverException(err)
raise Exception(err)
return ret
def force_scheduling_now():
"""Force scheduling some tasks now"""
get_all_backups.delay()
backup_running.delay()
get_all_clients_reports.delay()
@ns.route('/status/<task_id>', endpoint='async_restore_status')
@ns.doc(
params={
......@@ -287,7 +340,7 @@ class AsyncRestoreStatus(Resource):
db.session.commit()
task.revoke()
err = str(task.result)
self.abort(500, err)
self.abort(502, err)
if task.state == 'SUCCESS':
if not task.result:
self.abort(500, 'The task did not return anything')
......@@ -379,13 +432,13 @@ class AsyncGetFile(Resource):
mimetype='application/zip')
resp.set_cookie('fileDownload', 'true')
except Exception as e:
bui.cli.logger.error(str(e))
bui.client.logger.error(str(e))
self.abort(500, str(e))
return resp
def stream_file(self, path, filename, server):
socket = bui.cli.get_file(path, server)
socket = bui.client.get_file(path, server)
if not socket:
self.abort(500)
lengthbuf = socket.recv(8)
......@@ -723,6 +776,106 @@ class AsyncHistory(History):
# redirect to synchronous API call
# FIXME: Since we subclass the original code, we don't need the
# redirect anymore if the redirection is problematic
return redirect(url_for('api.history', client=client, server=server, start=args['start'], end=args['end']))
return redirect(
url_for(
'api.history',
client=client,
server=server,
start=args['start'],
end=args['end']
)
)
return self._get_backup_history(client, server, res)
@ns.route('/report',
'/<server>/report',
endpoint='async_clients_report')
@ns.doc(
params={
'server': 'Which server to collect data from when in multi-agent mode',
},
)
class AsyncClientsReport(ClientsReport):
"""The :class:`burpui.api.async.AsyncClientsReport` resource allows you to
access general reports about your clients.
This resource is part of the :mod:`burpui.api.clients` module.
An optional ``GET`` parameter called ``serverName`` is supported when
running in multi-agent mode.
"""
@api.cache.cached(timeout=1800, key_prefix=cache_key)
@ns.marshal_with(
ClientsReport.report_fields,
code=200,
description='Success',
strict=False
)
@ns.expect(ClientsReport.parser)
@ns.doc(
responses={
403: 'Insufficient permissions',
500: 'Internal failure',
},
)
def get(self, server=None):
"""Returns a global report about all the clients of a given server
**GET** method provided by the webservice.
The *JSON* returned is:
::
{
"backups": [
{
"name": "client1",
"number": 15
},
{
"name": "client2",
"number": 1
}
],
"clients": [
{
"name": "client1",
"stats": {
"total": 296377,
"totsize": 57055793698,
"windows": "unknown"
}
},
{
"name": "client2",
"stats": {
"total": 3117,
"totsize": 5345361,
"windows": "true"
}
}
]
}
The output is filtered by the :mod:`burpui.misc.acl` module so that you
only see stats about the clients you are authorized to.
:param server: Which server to collect data from when in multi-agent
mode
:type server: str
:returns: The *JSON* described above
"""
server = server or self.parser.parse_args()['serverName']
self._check_acl(server)
res = cache.cache.get('all_clients_reports')
if res is None:
# redirect to synchronous API call
# FIXME: Since we subclass the original code, we don't need the
# redirect anymore if the redirection is problematic
return redirect(url_for('api.clients_report', server=server))
return self._get_clients_reports(res, server)
burpui/api/backup.py
View file @ d19f6460
......@@ -70,7 +70,7 @@ class ServerBackup(Resource):
self.is_admin)):
self.abort(403, 'You are not allowed to access this client')
try:
return {'is_server_backup': bui.cli.is_server_backup(name, server)}
return {'is_server_backup': bui.client.is_server_backup(name, server)}
except BUIserverException as e:
self.abort(500, str(e))
......@@ -106,7 +106,7 @@ class ServerBackup(Resource):
self.is_admin)):
self.abort(403, 'You are not allowed to cancel a backup for this client')
try:
return bui.cli.cancel_server_backup(name, server)
return bui.client.cancel_server_backup(name, server)
except BUIserverException as e:
self.abort(500, str(e))
......@@ -147,7 +147,7 @@ class ServerBackup(Resource):
'You are not allowed to schedule a backup for this client'
)
try:
json = bui.cli.server_backup(name, server)
json = bui.client.server_backup(name, server)
return json, 201
except BUIserverException as e:
self.abort(500, str(e))
burpui/api/client.py
View file @ d19f6460
......@@ -16,12 +16,92 @@ from .custom.inputs import boolean
from ..exceptions import BUIserverException
from ..utils import NOTIF_ERROR
from six import iteritems
from flask_restplus.marshalling import marshal
from flask import current_app
bui = current_app # type: BUIServer
ns = api.namespace('client', 'Client methods')
node_fields = ns.model('ClientTree', {
'date': fields.DateTime(
required=True,
dt_format='iso8601',
description='Human representation of the backup date'
),
'gid': fields.Integer(
required=True,
description='gid owner of the node'
),
'inodes': fields.Integer(
required=True,
description='Inodes of the node'
),
'mode': fields.String(
required=True,
description='Human readable mode. Example: "drwxr-xr-x"'
),
'name': fields.String(
required=True,
description='Node name'
),
'title': fields.SafeString(
required=True,
description='Node name (alias)',
attribute='name'
),
'fullname': fields.String(
required=True,
description='Full name of the Node'
),
'key': fields.String(
required=True,
description='Full name of the Node (alias)',
attribute='fullname'
),
'parent': fields.String(
required=True,
description='Parent node name'
),
'size': fields.String(
required=True,
description='Human readable size. Example: "12.0KiB"'
),
'type': fields.String(
required=True,
description='Node type. Example: "d"'
),
'uid': fields.Integer(
required=True,
description='uid owner of the node'
),
'selected': fields.Boolean(
required=False,
description='Is path selected',
default=False
),
'lazy': fields.Boolean(
required=False,
description='Do the children have been loaded during this' +
' request or not',
default=True
),
'folder': fields.Boolean(
required=True,
description='Is it a folder'
),
'expanded': fields.Boolean(
required=False,
description='Should we expand the node',
default=False
),
# Cannot use nested on own
'children': fields.Raw(
required=False,
description='List of children'
),
})
@ns.route('/browse/<name>/<int:backup>',
'/<server>/browse/<name>/<int:backup>',
......@@ -72,83 +152,6 @@ class ClientTree(Resource):
required=False,
default=False
)
node_fields = ns.model('ClientTree', {
'date': fields.DateTime(
required=True,
dt_format='iso8601',
description='Human representation of the backup date'
),
'gid': fields.Integer(
required=True,
description='gid owner of the node'
),
'inodes': fields.Integer(
required=True,
description='Inodes of the node'
),
'mode': fields.String(
required=True,
description='Human readable mode. Example: "drwxr-xr-x"'
),
'name': fields.String(
required=True,
description='Node name'
),
'title': fields.SafeString(
required=True,
description='Node name (alias)',
attribute='name'
),
'fullname': fields.String(
required=True,
description='Full name of the Node'
),
'key': fields.String(
required=True,
description='Full name of the Node (alias)',
attribute='fullname'
),
'parent': fields.String(
required=True,
description='Parent node name'
),
'size': fields.String(
required=True,
description='Human readable size. Example: "12.0KiB"'
),
'type': fields.String(
required=True,
description='Node type. Example: "d"'
),
'uid': fields.Integer(
required=True,
description='uid owner of the node'
),
'selected': fields.Boolean(
required=False,
description='Is path selected',
default=False
),
'lazy': fields.Boolean(
required=False,
description='Do the children have been loaded during this' +
' request or not',
default=True
),
'folder': fields.Boolean(
required=True,
description='Is it a folder'
),
'expanded': fields.Boolean(
required=False,
description='Should we expand the node',
default=False
),
'children': fields.Raw(
required=False,
description='List of children'
),
})
@api.cache.cached(timeout=3600, key_prefix=cache_key)
@ns.marshal_list_with(node_fields, code=200, description='Success')
......@@ -229,7 +232,7 @@ class ClientTree(Resource):
path = ''
# fetch the root first if not already loaded
if not root_loaded:
part = bui.cli.get_tree(
part = bui.client.get_tree(
name,
backup,
level=0,
......@@ -256,7 +259,7 @@ class ClientTree(Resource):
path = '/'
if path in paths_loaded:
continue
temp = bui.cli.get_tree(
temp = bui.client.get_tree(
name,
backup,
path,
......@@ -266,7 +269,7 @@ class ClientTree(Resource):
paths_loaded.append(path)
part += temp
else:
part = bui.cli.get_tree(
part = bui.client.get_tree(
name,
backup,
root,
......@@ -284,7 +287,7 @@ class ClientTree(Resource):
entry['selected'] = True
if not root_list:
json = bui.cli.get_tree(name, backup, agent=server)
json = bui.client.get_tree(name, backup, agent=server)
if args['selected']:
for entry in json:
if not entry['parent']:
......@@ -296,9 +299,9 @@ class ClientTree(Resource):
roots = []
for entry in json:
# /!\ after marshalling, 'fullname' will be 'key'
tree[entry['fullname']] = marshal(entry, self.node_fields)
tree[entry['fullname']] = marshal(entry, node_fields)
for key, entry in tree.iteritems():
for key, entry in iteritems(tree):
parent = entry['parent']
if not entry['children']:
entry['children'] = None
......@@ -328,6 +331,168 @@ class ClientTree(Resource):
return json
@ns.route('/browseall/<name>/<int:backup>',
'/<server>/browseall/<name>/<int:backup>',
endpoint='client_tree_all')
@ns.doc(
params={
'server': 'Which server to collect data from when in' +
' multi-agent mode',
'name': 'Client name',
'backup': 'Backup number',
},
)
class ClientTreeAll(Resource):
"""The :class:`burpui.api.client.ClientTreeAll` resource allows you to
retrieve a list of all the files in a given backup.
This resource is part of the :mod:`burpui.api.client` module.
An optional ``GET`` parameter called ``serverName`` is supported when
running in multi-agent mode.
"""
parser = ns.parser()
parser.add_argument(
'serverName',
help='Which server to collect data from when in multi-agent mode'
)
@api.cache.cached(timeout=3600, key_prefix=cache_key)
@ns.marshal_list_with(node_fields, code=200, description='Success')
@ns.expect(parser)
@ns.doc(
responses={
'403': 'Insufficient permissions',
'405': 'Method not allowed',
'500': 'Internal failure',
},
)
def get(self, server=None, name=None, backup=None):
"""Returns a list of all 'nodes' of a given backup
**GET** method provided by the webservice.
The *JSON* returned is:
::
[
{
"date": "2015-05-21 14:54:49",
"gid": "0",
"inodes": "173",
"selected": false,
"expanded": false,
"children": [],
"mode": "drwxr-xr-x",
"name": "/",
"key": "/",
"title": "/",
"fullname": "/",
"parent": "",
"size": "12.0KiB",
"type": "d",
"uid": "0"
}
]
The output is filtered by the :mod:`burpui.misc.acl` module so that you
only see stats about the clients you are authorized to.
:param server: Which server to collect data from when in multi-agent
mode
:type server: str
:param name: The client we are working on
:type name: str
:param backup: The backup we are working on
:type backup: int
:returns: The *JSON* described above.
"""
args = self.parser.parse_args()
server = server or args['serverName']
if not bui.client.get_attr('batch_list_supported', False, server):
self.abort(
405,
'Sorry, the requested backend does not support this method'
)
if (bui.acl and
(not self.is_admin and not
bui.acl.is_client_allowed(self.username,
name,
server))):
self.abort(403, 'Sorry, you are not allowed to view this client')
try:
json = bui.client.get_tree(name, backup, '*', agent=server)
tree = {}
rjson = []
roots = []
def __expand_json(js):
res = {}
for entry in js:
# /!\ after marshalling, 'fullname' will be 'key'
res[entry['fullname']] = marshal(entry, node_fields)
return res
tree = __expand_json(json)
# TODO: we can probably improve this at some point
redo = True
while redo:
redo = False
for key, entry in iteritems(tree):
parent = entry['parent']
if not entry['children']:
entry['children'] = None
if parent:
if parent not in tree:
parent2 = parent
last = False
while parent not in tree and not last:
if not parent2:
last = True
json = bui.client.get_tree(
name,
backup,
parent2,
agent=server
)
if parent2 == '/':
parent2 = ''
else:
parent2 = os.path.dirname(parent2)
tree2 = __expand_json(json)
tree.update(tree2)
roots = []
redo = True
break
node = tree[parent]
if not node['children']:
node['children'] = []
elif entry in node['children']:
continue
node['children'].append(entry)
if node['folder']:
node['lazy'] = False
node['expanded'] = False
else:
roots.append(entry['key'])
for fullname in roots:
rjson.append(tree[fullname])
json = rjson
except BUIserverException as e:
self.abort(500, str(e))
return json
@ns.route('/report/<name>',
'/<server>/report/<name>',
'/report/<name>/<int:backup>',
......@@ -626,19 +791,19 @@ class ClientReport(Resource):
self.abort(403, 'You don\'t have rights to view this client report')
if backup:
try:
j = bui.cli.get_backup_logs(backup, name, agent=server)
j = bui.client.get_backup_logs(backup, name, agent=server)
except BUIserverException as e:
self.abort(500, str(e))
else:
try:
cl = bui.cli.get_client(name, agent=server)
cl = bui.client.get_client(name, agent=server)
except BUIserverException as e:
self.abort(500, str(e))
err = []
for c in cl:
try:
j.append(
bui.cli.get_backup_logs(
bui.client.get_backup_logs(
c['number'],
name,
agent=server
......@@ -744,7 +909,7 @@ class ClientStats(Resource):
name,
server))):
self.abort(403, 'Sorry, you cannot access this client')
j = bui.cli.get_client(name, agent=server)
j = bui.client.get_client(name, agent=server)
except BUIserverException as e:
self.abort(500, str(e))
return j
burpui/api/clients.py
View file @ d19f6460
......@@ -76,14 +76,14 @@ class RunningClients(Resource):
server)):
r = []
return r
if bui.cli.is_backup_running(client, server):
if bui.client.is_backup_running(client, server):
r = [client]
return r
else:
r = []
return r
r = bui.cli.is_one_backup_running(server)
r = bui.client.is_one_backup_running(server)
# Manage ACL
if bui.acl and not self.is_admin:
if isinstance(r, dict):
......@@ -141,7 +141,7 @@ class RunningBackup(Resource):
"""
return {
'running': self._is_one_backup_running(
bui.cli.is_one_backup_running(server),
bui.client.is_one_backup_running(server),
server
)
}
......@@ -190,6 +190,15 @@ class ClientsReport(Resource):
"""
parser = ns.parser()
parser.add_argument('serverName', help='Which server to collect data from when in multi-agent mode')
parser.add_argument('limit', type=int, default=8, help='Number of elements to return')
parser.add_argument('aggregation', help='What aggregation to operate', default='number', choices=('number', 'files', 'size'))
translation = {
'number': 'number',
'files': 'total',
'size': 'totsize',
}
stats_fields = ns.model('ClientsStats', {
'total': fields.Integer(required=True, description='Number of files', default=0),
'totsize': fields.Integer(required=True, description='Total size occupied by all the backups of this client', default=0),
......@@ -267,23 +276,102 @@ class ClientsReport(Resource):
"""
server = server or self.parser.parse_args()['serverName']
j = {}
self._check_acl(server)
return self._get_clients_reports(server=server)
def _check_acl(self, server):
# Manage ACL
if (not bui.standalone and bui.acl and
(not self.is_admin and
server not in
bui.acl.servers(self.username))):
self.abort(403, 'Sorry, you don\'t have any rights on this server')
clients = []
if bui.acl and not self.is_admin:
clients = [{'name': x} for x in bui.acl.clients(self.username, server)]
def _get_clients_reports(self, res=None, server=None):
args = self.parser.parse_args()
limit = args['limit']
aggregation = self.translation.get(args['aggregation'], 'number')
ret = self._parse_clients_reports(res, server)
backups = backups_orig = ret.get('backups', [])
clients = clients_orig = ret.get('clients', [])
aggregate = False
if limit > 1:
limit -= 1
aggregate = True
# limit the number of elements to return so the graphs stay readable
if len(backups) > limit and limit > 0:
if aggregation == 'number':
backups = (
sorted(backups, key=lambda x: x.get('number'), reverse=True)
)[:limit]
else:
clients = (
sorted(clients, key=lambda x: x.get('stats', {}).get(aggregation), reverse=True)
)[:limit]
else:
try:
clients = bui.cli.get_all_clients(agent=server)
except BUIserverException as e:
self.abort(500, str(e))
j = bui.cli.get_clients_report(clients, server)
return j
aggregate = False
if aggregation == 'number':
clients_name = [x.get('name') for x in backups]
ret['backups'] = backups
ret['clients'] = [
x for x in clients_orig
if x.get('name') in clients_name
]
else:
clients_name = [x.get('name') for x in clients]
ret['clients'] = clients
ret['backups'] = [
x for x in backups_orig
if x.get('name') in clients_name
]
if aggregate:
backups = {'name': 'others', 'number': 0}
for client in backups_orig:
if client.get('name') not in clients_name:
backups['number'] += client.get('number', 0)
complement = {
'name': 'others',
'stats': {
'total': 0,
'totsize': 0,
'windows': None
}
}
for client in clients_orig:
if client.get('name') not in clients_name:
complement['stats']['total'] += client.get('stats', {}).get('total', 0)
complement['stats']['totsize'] += client.get('stats', {}).get('totsize', 0)
os = client.get('stats', {}).get('windows', 'unknown')
if not complement['stats']['windows']:
complement['stats']['windows'] = os
elif os != complement['stats']['windows']:
complement['stats']['windows'] = 'unknown'
ret['clients'].append(complement)
ret['backups'].append(backups)
return ret
def _parse_clients_reports(self, res=None, server=None):
if not res:
clients = []
if bui.acl and not self.is_admin:
clients = [{'name': x} for x in bui.acl.clients(self.username, server)]
else:
try:
clients = bui.client.get_all_clients(agent=server)
except BUIserverException as e:
self.abort(500, str(e))
return bui.client.get_clients_report(clients, server)
if bui.standalone:
ret = res
else:
ret = res.get(server, {})
if bui.acl and not self.is_admin:
ret['backups'] = [x for x in ret.get('backups', []) if bui.acl.is_client_allowed(self.username, x.get('name'), server)]
ret['clients'] = [x for x in ret.get('clients', []) if bui.acl.is_client_allowed(self.username, x.get('name'), server)]
return ret
@ns.route('/stats',
......@@ -309,7 +397,7 @@ class ClientsStats(Resource):
'last': fields.DateTime(required=True, dt_format='iso8601', description='Date of last backup'),
'human': fields.DateTimeHuman(required=True, attribute='last', description='Human readable date of the last backup'),
'name': fields.String(required=True, description='Client name'),
'state': fields.String(required=True, description='Current state of the client (idle, backup, etc.)'),
'state': fields.LocalizedString(required=True, description='Current state of the client (idle, backup, etc.)'),
'phase': fields.String(description='Phase of the current running backup'),
'percent': fields.Integer(description='Percentage done', default=0),
})
......@@ -368,7 +456,7 @@ class ClientsStats(Resource):
server not in
bui.acl.servers(self.username))):
self.abort(403, 'Sorry, you don\'t have any rights on this server')
j = bui.cli.get_all_clients(agent=server)
j = bui.client.get_all_clients(agent=server)
if bui.acl and not self.is_admin:
j = [x for x in j if x['name'] in bui.acl.clients(self.username, server)]
except BUIserverException as e:
......@@ -449,7 +537,7 @@ class AllClients(Resource):
self.abort(403, "You are not allowed to view this server infos")
if server:
clients = bui.cli.get_all_clients(agent=server)
clients = bui.client.get_all_clients(agent=server)
if bui.acl and not self.is_admin:
ret = [{'name': x, 'agent': server} for x in bui.acl.clients(self.username, server)]
else:
......@@ -460,18 +548,18 @@ class AllClients(Resource):
if bui.acl and not self.is_admin:
ret = [{'name': x} for x in bui.acl.clients(self.username)]
else:
ret = [{'name': x['name']} for x in bui.cli.get_all_clients()]
ret = [{'name': x['name']} for x in bui.client.get_all_clients()]
else:
grants = {}
if bui.acl and not self.is_admin:
for serv in bui.acl.servers(self.username):
grants[serv] = bui.acl.clients(self.username, serv)
else:
for serv in bui.cli.servers:
for serv in bui.client.servers:
grants[serv] = 'all'
for (serv, clients) in iteritems(grants):
if not isinstance(clients, list):
clients = [x['name'] for x in bui.cli.get_all_clients(agent=serv)]
clients = [x['name'] for x in bui.client.get_all_clients(agent=serv)]
ret += [{'name': x, 'agent': serv} for x in clients]
return ret