burp-ui cache issues + remove gevent calls
Hi,
You told me to talk to you again about caching issues if I see some.
Updated to latest v0.7.0-dev from today. I had my bui-agent service that didn't want to run in the first place (some stupid error I made), so I had Burp-UI not showing my servers as Status 'X'.
Fixing my bui-agent service file, my session still shows now my server status as 'X', it's been a some minutes.
[EDIT] After having finished writing this issue, I updated my browser window, and everything seems to work now... Cache expired ?[/EDIT]
Opening my browser in incognito mode, I was able to click on the server (which status was fine) and browse among backups until I tried to open a particular backup which gave me the following:
cannot switch to a different thread
bui-agent corresponding log:
[2019-07-30 09:30:40,184] INFO in agent.handle: recv: b'{"func": "get_all_clients", "args": {}, "password": "somepass"}'
[2019-07-30 09:30:40,185] INFO in burp2.status: 140300300577704 - query: 'c:' (cache: True, raw: False)
[2019-07-30 09:30:40,186] INFO in agent.handle: recv: b'{"func": "get_all_clients", "args": {}, "password": "somepass"}'
[2019-07-30 09:30:40,187] ERROR in agent.handle: cannot switch to a different thread
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/burpui_agent/engines/agent.py", line 293, in handle
res = json.dumps(await trio.run_sync_in_worker_thread(callback))
File "/usr/local/lib/python3.6/site-packages/burpui_agent/../trio/_threads.py", line 400, in run_sync_in_worker_thread
return await _core.wait_task_rescheduled(abort)
File "/usr/local/lib/python3.6/site-packages/burpui_agent/../trio/_core/_traps.py", line 166, in wait_task_rescheduled
return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
File "/usr/local/lib/python3.6/site-packages/burpui_agent/../outcome/_sync.py", line 111, in unwrap
raise captured_error
File "/usr/local/lib/python3.6/site-packages/burpui_agent/../trio/_threads.py", line 362, in do_release_then_return_result
return result.unwrap()
File "/usr/local/lib/python3.6/site-packages/burpui_agent/../outcome/_sync.py", line 111, in unwrap
raise captured_error
File "/usr/local/lib/python3.6/site-packages/burpui_agent/misc/backend/burp2.py", line 604, in get_all_clients
query = self.status()
File "/usr/local/lib/python3.6/site-packages/burpui_agent/misc/backend/burp2.py", line 137, in status
with self.plock:
File "/usr/local/lib64/python3.6/site-packages/gevent/lock.py", line 230, in __enter__
return self.acquire()
File "/usr/local/lib64/python3.6/site-packages/gevent/lock.py", line 223, in acquire
rc = self._block.acquire(blocking)
File "src/gevent/_semaphore.py", line 100, in gevent.__semaphore.Semaphore.acquire
File "src/gevent/_semaphore.py", line 128, in gevent.__semaphore.Semaphore.acquire
File "src/gevent/_abstract_linkable.py", line 192, in gevent.__abstract_linkable.AbstractLinkable._wait
File "src/gevent/_abstract_linkable.py", line 165, in gevent.__abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_abstract_linkable.py", line 169, in gevent.__abstract_linkable.AbstractLinkable._wait_core
File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 64, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/__greenlet_primitives.pxd", line 35, in gevent.__greenlet_primitives._greenlet_switch
greenlet.error: cannot switch to a different thread
[2019-07-30 09:30:40,188] WARNING in agent.handle: Forwarding Exception: cannot switch to a different thread
Using another incognito window, everything works fine and I can browse the same backup. So I guess there's still some caching issues, cannot diagnose what they are and where they come from though. My best guess is that the cache isn't validated against changes.
Burp
$ burp -v
burp-2.3.6
Sysinfo
$ bui-manage sysinfo
/usr/local/share/burpui/migrations
{'sqlalchemy': <flask_sqlalchemy._SQLAlchemyState object at 0x7f3d8656a7b8>}
Python version: 3.6.8
Burp-UI version: 0.7.0.dev0 ()
OS: Linux:3.10.0-957.21.3.el7.x86_64 (posix)
Distribution: centos 7.6.1810 Core
Single mode: False
Backend: multi
WebSocket embedded: False
WebSocket available: True
Config file: /etc/burp/burpui.cfg
Steps to reproduce (hopefully)
- Have an agent setup
- Disable agent service
- Login and see server status 'x'
- Enable agent service again
- Update view, there are changes server status remains 'x'
Configuration
[Global]
backend = multi
auth = basic
acl = basic
audit = basic
plugins = none
[UI]
refresh = 180
liverefresh = 5
ignore_labels = color:.*, custom:.*
format_labels = s/^os:\s*//
default_strip = 0
[Production]
storage = redis
session = redis
cache = redis
redis = localhost:6379
celery = true
database = sqlite:////var/lib/burpui/store.db
limiter = false
ratio = 60/minute
prefix = none
num_proxies = 0
proxy_fix_args = "{'x_for': {num_proxies}, 'x_host': {num_proxies}, 'x_prefix': {num_proxies}}"
[WebSocket]
enabled = false
embedded = false
broker = redis
url = none
debug = false
[Security]
includes = /etc/burp
enforce = false
revoke = true
cookietime = 1
sessiontime = 1
scookie = true
appsecret = somestring
[Experimental]
zip64 = false
noserverrestore = false
[Burp]
burpbin = /usr/sbin/burp
stripbin = /usr/sbin/vss_strip
bconfcli = /etc/burp/burp.conf
bconfsrv = /etc/burp/burp-server.conf
tmpdir = /tmp/bui
timeout = 15
deep_inspection = false
[Parallel]
host = ::1
port = 11111
timeout = 15
password = password123456
ssl = true
concurrency = 2
init_wait = 15
[BASIC:AUDIT]
priority = 100
level = WARNING
logfile = /var/log/burp-ui.log
max_bytes = 30 * 1024 * 1024
rotate = 5
[BASIC:AUTH]
priority = 100
admin = pbkdf2:sha256:somesalt
[ACL]
extended = true
assume_rw = true
legacy = false
[BASIC:ACL]
priority = 100
admin = admin
supadmin = '{"agents":{"Agent1":{"rw":["client.*","server.*"]},"Agent2":{"rw":["client.*","server.*"]}}}'
[Agent:Agent1]
host = 127.0.0.1
port = 10000
password = somepass
ssl = false
timeout = 300
[Agent:Agent1]
host = 127.0.0.1
port = 10001
password = somepass
ssl = false
timeout = 300
Thanks