remote debugger pathMappings issue? is it possible to add more than one mapping?

[gerardosabetta]

Hi I'm struggling to add more than one thing to the pathMapping settings when configuring a remote debugger.

This how my coworkers have their pycharm set up

So I thought the equivalent would be something like this

But it's not working as expected, the breakpoint for the file on the right triggered an error saying "api.py does not exist, do you want to create it" and it was created on the path you can see on the left 👇

My initial thought was "it seems like you can only do one map" but this can't be the case, pathMappings expects an array. I must be doing something wrong.

Thanks in advance

[karthiknadig]

@gerardosabetta What you have there seems right.

@int19h, @fabioz : Do we log path mapping issues when we enable logs via DAP? Also, should this be moved to debugpy?

[int19h]

I wonder if perhaps we inadvertently regressed the propagation of logging level - pydevd actually has its own setting, but debugpy is supposed to set it when it initializes its own logging, yet in your case it clearly did so for its logs but not for pydevd. I'll look into it, but in the meantime, try enabling it for pydevd directly with these env vars:

PYDEVD_DEBUG=True
PYDEVD_DEBUG_FILE=.../pydevd.log 

(Note that in this case, you're specifying the name of the file that'll write, rather than a directory in which files are to be created.)

[gerardosabetta]

There we go, now they are all logging to the same directory. However I'm getting a lot of logs that go File traced: ... or File not traced: ... and they pretty much disclose the whole repo I don't think I'm allowed to share that.

for the file I used in the example it's logging

0.00s - File traced: /opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py

Can you point me to what we are looking for without me uploading sensitive info to github?

[gerardosabetta]

Is this what we are after?

0.00s - Process InitializeRequest: {
    "arguments": {
        "adapterID": "python",
        "clientID": "vscode",
        "clientName": "Visual Studio Code",
        "columnsStartAt1": true,
        "linesStartAt1": true,
        "locale": "en",
        "pathFormat": "path",
        "supportsInvalidatedEvent": true,
        "supportsMemoryReferences": true,
        "supportsProgressReporting": true,
        "supportsRunInTerminalRequest": true,
        "supportsVariablePaging": true,
        "supportsVariableType": true
    },
    "command": "initialize",
    "seq": 9,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 9, "success": true, "command": "initialize", "seq": 24, "body": {"supportsConfigurationDoneRequest": true, "supportsFunctionBreakpoints": true, "supportsConditionalBreakpoints": true, "supportsHitConditionalBreakpoints": true, "supportsEvaluateForHovers": true, "exceptionBreakpointFilters": [{"filter": "raised", "label": "Raised Exceptions", "default": false}, {"filter": "uncaught", "label": "Uncaught Exceptions", "default": true}, {"filter": "userUnhandled", "label": "User Uncaught Exceptions", "default": false}], "supportsStepBack": false, "supportsSetVariable": true, "supportsRestartFrame": false, "supportsGotoTargetsRequest": true, "supportsStepInTargetsRequest": true, "supportsCompletionsRequest": true, "completionTriggerCharacters": [], "supportsModulesRequest": true, "additionalModuleColumns": [], "supportedChecksumAlgorithms": [], "supportsRestartRequest": false, "supportsExceptionOptions": true, "supportsValueFormattingOptions": true, "supportsExceptionInfoRequest": true, "supportTerminateDebuggee": true, "supportsDelayedStackTraceLoading": true, "supportsLoadedSourcesRequest": false, "supportsLogPoints": true, "supportsTerminateThreadsRequest": false, "supportsSetExpression": true, "supportsTerminateRequest": true, "supportsDataBreakpoints": false, "supportsReadMemoryRequest": false, "supportsDisassembleRequest": false, "supportsClipboardContext": true, "supportsDebuggerProperties": true, "pydevd": {"processId": 125}}, "pydevd_cmd_id": 502}

0.00s - Process AttachRequest: {
    "arguments": {
        "__configurationTarget": 5,
        "__sessionId": "3f8a918f-3673-45e0-8a4b-e2bbdb7d3a43",
        "connect": {
            "host": "localhost",
            "port": 3000
        },
        "debugOptions": [
            "RedirectOutput",
            "UnixClient",
            "ShowReturnValue"
        ],
        "justMyCode": true,
        "logToFile": true,
        "name": "Subprocess 125",
        "pathMappings": [
            {
                "localRoot": "/home/gera/dev/platform-backend/firedrill-data",
                "remoteRoot": "/opt/attackiq/firedrill-data"
            },
            {
                "localRoot": "/home/gera/dev/platform-backend/ai_cloud_foundation/ai_cloud_foundation",
                "remoteRoot": "/opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation"
            }
        ],
        "request": "attach",
        "showReturnValue": true,
        "subProcessId": 125,
        "type": "python",
        "workspaceFolder": "/home/gera/dev/platform-backend"
    },
    "command": "attach",
    "seq": 10,
    "type": "request"
}

0.00s - sending cmd (http_json) -->    CMD_PROCESS_EVENT {"type": "event", "event": "process", "body": {"name": "/opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/manage.py", "systemProcessId": 125, "isLocalProcess": true, "startMethod": "attach"}, "seq": 26, "pydevd_cmd_id": 204}

0.00s - pydevd: Use libraries filter: True

0.00s - Show return values: True
0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 10, "success": true, "command": "attach", "seq": 28, "pydevd_cmd_id": 502}

0.00s - Process SetDebuggerPropertyRequest: {
    "arguments": {
        "ideOS": "UNIX",
        "multiThreadsSingleNotification": true,
        "skipPrintBreakpointException": [
            "NameError"
        ],
        "skipSuspendOnBreakpointException": [
            "BaseException"
        ]
    },
    "command": "setDebuggerProperty",
    "seq": 11,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 11, "success": true, "command": "setDebuggerProperty", "seq": 30, "body": {}, "pydevd_cmd_id": 502}

0.02s - Process SetBreakpointsRequest: {
    "arguments": {
        "breakpoints": [
            {
                "line": 26
            }
        ],
        "lines": [
            26
        ],
        "source": {
            "name": "api.py",
            "path": "/home/gera/dev/platform-backend/ai_cloud_foundation/ai_cloud_foundation/ai_cloud/api/phase/api.py"
        },
        "sourceModified": false
    },
    "command": "setBreakpoints",
    "seq": 12,
    "type": "request"
}

0.00s - Request for breakpoint in: /home/gera/dev/platform-backend/ai_cloud_foundation/ai_cloud_foundation/ai_cloud/api/phase/api.py line: 26
0.00s - Breakpoint (after path translation) in: /opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py line: 26
0.00s - Added breakpoint:/opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py - line:26 - func_name:None

[int19h]

Yep, that's the first interesting bit. But there should also be a similar one "in reverse" for when it actually hits the breakpoint- look for PausedEvent in that log, it should be somewhere right before it.

@fabioz, is there anything here that looks off to you?

[gerardosabetta]

I don't see any matches when searching for PausedEvents but I'm looking at the logs live and when I hit the endpoint where I have set my breakpoint the logs that are being generated right at that moment are

15.53s - sending cmd (http_json) -->    CMD_THREAD_CREATE {"type": "event", "event": "thread", "body": {"reason": "started", "threadId": 37}, "seq": 224, "pydevd_cmd_id": 103}

0.01s - Suspending all threads except: <Thread(Thread-9 (process_request_thread), started daemon 140396018529856)>
0.00s - Set tracing of frame: /usr/local/lib/python3.10/dist-packages/django/utils/autoreload.py - tick
0.00s - Set tracing of frame: /usr/local/lib/python3.10/dist-packages/django/utils/autoreload.py - run_loop
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/_pydev_bundle/_pydev_execfile.py - execfile
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/pydevd.py - _exec
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/pydevd.py - run
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/pydevd.py - main
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/pydevd.py - <module>
0.00s - Set tracing of frame: /usr/lib/python3.10/selectors.py - select
0.00s - Set tracing of frame: /usr/lib/python3.10/socketserver.py - serve_forever
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - run
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - _bootstrap_inner
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - _bootstrap
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py - __call__
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/socket.py - readinto
0.00s - Set tracing of frame: /usr/local/lib/python3.10/dist-packages/django/core/servers/basehttp.py - handle_one_request
0.00s - Set tracing of frame: /usr/local/lib/python3.10/dist-packages/django/core/servers/basehttp.py - handle
0.00s - Set tracing of frame: /usr/lib/python3.10/socketserver.py - __init__
0.00s - Set tracing of frame: /usr/lib/python3.10/socketserver.py - finish_request
0.00s - Set tracing of frame: /usr/lib/python3.10/socketserver.py - process_request_thread
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - run
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - _bootstrap_inner
0.00s - SKIP set tracing of frame: /usr/lib/python3.10/threading.py - _bootstrap
0.00s - SKIP set tracing of frame: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py - __call__
0.00s - PyDB.do_wait_suspend
name: list (line: 26)
 file: /opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py
 event: line
 arg: None
 step: CMD_STEP_INTO (original step: <Unknown: -1>)
 thread: <Thread(Thread-9 (process_request_thread), started daemon 140396018529856)>, thread id: pid_125_id_140396020280448, id(thread): 140396020280448
0.00s -   Stack: /opt/attackiq/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py, list, 26
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/rest_framework/views.py, dispatch, 506
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/rest_framework/viewsets.py, view, 125
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/views/decorators/csrf.py, wrapped_view, 54
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/base.py, _get_response, 181
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/utils/deprecation.py, __call__, 117
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py, inner, 47
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/base.py, get_response, 130
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/handlers/wsgi.py, __call__, 133
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/contrib/staticfiles/handlers.py, __call__, 76
0.00s -   Stack: /usr/lib/python3.10/wsgiref/handlers.py, run, 137
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/servers/basehttp.py, handle_one_request, 201
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/django/core/servers/basehttp.py, handle, 178
0.00s -   Stack: /usr/lib/python3.10/socketserver.py, __init__, 747
0.00s -   Stack: /usr/lib/python3.10/socketserver.py, finish_request, 360
0.00s -   Stack: /usr/lib/python3.10/socketserver.py, process_request_thread, 683
0.00s -   Stack: /usr/lib/python3.10/threading.py, run, 946
0.00s -   Stack: /usr/lib/python3.10/threading.py, _bootstrap_inner, 1009
0.00s -   Stack: /usr/lib/python3.10/threading.py, _bootstrap, 966
0.00s -   Stack: /usr/local/lib/python3.10/dist-packages/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py, __call__, 1044
0.00s - Sending suspend notification.
0.00s - sending cmd (http_json) --> CMD_THREAD_SUSPEND_SINGLE_NOTIFICATION {"type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 12, "preserveFocusHint": false, "allThreadsStopped": true}, "seq": 226, "pydevd_cmd_id": 157}

0.01s - Process ThreadsRequest: {
    "command": "threads",
    "seq": 58,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 58, "success": true, "command": "threads", "body": {"threads": [{"id": 1, "name": "MainThread"}, {"id": 2, "name": "django-main-thread"}, {"id": 12, "name": "Thread-9 (process_request_thread)"}, {"id": 37, "name": "Thread-13 (process_request_thread)"}]}, "seq": 228, "pydevd_cmd_id": 502}

0.03s - Process StackTraceRequest: {
    "arguments": {
        "format": {},
        "levels": 1,
        "startFrame": 0,
        "threadId": 12
    },
    "command": "stackTrace",
    "seq": 59,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 59, "success": true, "command": "stackTrace", "body": {"stackFrames": [{"id": 5, "name": "list", "line": 26, "column": 1, "source": {"path": "/home/gera/dev/platform-backend/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py", "sourceReference": 0}}], "totalFrames": 1}, "seq": 230, "pydevd_cmd_id": 502}

0.41s - Process ThreadsRequest: {
    "command": "threads",
    "seq": 60,
    "type": "request"
}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 60, "success": true, "command": "threads", "body": {"threads": [{"id": 1, "name": "MainThread"}, {"id": 2, "name": "django-main-thread"}, {"id": 12, "name": "Thread-9 (process_request_thread)"}, {"id": 37, "name": "Thread-13 (process_request_thread)"}]}, "seq": 232, "pydevd_cmd_id": 502}

After hitting continue on the debugger it logs:

73.49s - Process ContinueRequest: {
    "arguments": {
        "threadId": "*"
    },
    "command": "continue",
    "seq": 64,
    "type": "request"
}

0.00s - Resuming threads: * (except thread: None)
0.00s - Sending resume notification.
0.00s - sending cmd (http_json) --> CMD_THREAD_RESUME_SINGLE_NOTIFICATION {"type": "event", "event": "continued", "body": {"threadId": 12, "allThreadsContinued": true}, "seq": 240, "pydevd_cmd_id": 158}

0.00s - sending cmd (http_json) -->           CMD_RETURN {"type": "response", "request_seq": 64, "success": true, "command": "continue", "body": {"allThreadsContinued": true}, "seq": 242, "pydevd_cmd_id": 502}

0.00s - Leaving PyDB.do_wait_suspend: <Thread(Thread-9 (process_request_thread), started daemon 140396018529856)> (pid_125_id_140396020280448) 140396020280448
0.00s - sending cmd (http_json) --> CMD_WRITE_TO_CONSOLE {"type": "event", "event": "output", "body": {"output": "test", "category": "stdout", "source": {}}, "seq": 244, "pydevd_cmd_id": 116}

0.00s - sending cmd (http_json) --> CMD_WRITE_TO_CONSOLE {"type": "event", "event": "output", "body": {"output": "\n", "category": "stdout", "source": {}}, "seq": 246, "pydevd_cmd_id": 116}

0.20s - sending cmd (http_json) -->      CMD_THREAD_KILL {"type": "event", "event": "thread", "body": {"reason": "exited", "threadId": 37}, "seq": 248, "pydevd_cmd_id": 104}

in case that's relevant

[int19h]

@gerardosabetta Can you try hovering over the tab with incorrectly mapped path, and check what the tooltip says the path is.

[gerardosabetta]

The one on the left? Yes, let me reproduce that, but I think you can infer the path from the breadcrumbs bellow the tab. It' should be ${workspaceFolder}/firedrill-data/ai_py/firedrill/ai_django/ai_cloud_foundation/ai_cloud/api/phase/api.py

But let me try and reproduce it so I can attach a screenshot showing the actual absolute path

[gerardosabetta]

That's the notification I get, although the breakpoint is reached and the code stops running the mapping is not right.

If I press "Create file" it writes that file to this location

Which is the same as the comment I left earlier

[ced-mos]

+1 for me, as I experience the same behavior . Only the first entry in the pathMappings is used, although it supports a list of entries.

My system used:

Version: 1.67.0
Commit: 57fd6d0195bb9b9d1b49f6da5db789060795de47
Date: 2022-05-04T12:08:42.978Z
Electron: 17.4.1
Chromium: 98.0.4758.141
Node.js: 16.13.0
V8: 9.8.177.13-electron.0
OS: Linux x64 5.13.0-40-generic snap

Have you already tracked down where the issue could be to file a bug report?

[fabioz]

@ced-mos please create a separate issue in https://github.com/microsoft/debugpy/issues providing the pydevd logs (which are needed to diagnose this issue).