Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failure to substitute :method (with GET?) sometimes in MitmRequestAgent #368

Open
andynuss opened this issue Oct 31, 2021 · 1 comment
Open

Comments

@andynuss
Copy link

andynuss commented Oct 31, 2021

I don't know if this has anything to do with this issue (ulixee/hero#161), but
on one of the random urls that I was testing with happened to be a PDF, and my default browser plugin
seemingly failed to substitute the :method template param with GET, before issuing the GET, I assume
to download the PDF. I only saw it happen once so far but will keep a watch on it. I'm betting it had something to do with my low, horse-powered ec2 instance
that I am testing on.

TypeError [ERR_INVALID_HTTP_TOKEN]: Header name must be a valid HTTP token [":method"]
    at ClientRequest.setHeader (_http_outgoing.js:563:3)
    at new ClientRequest (_http_client.js:262:14)
    at Object.request (https.js:370:10)
    at MitmRequestAgent.http1Request (/home/ec2-user/app/node_modules/mitm/lib/MitmRequestAgent.ts:217:32)
    at MitmRequestAgent.request (/home/ec2-user/app/node_modules/mitm/lib/MitmRequestAgent.ts:92:17)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at HttpRequestHandler.createProxyToServerRequest (/home/ec2-user/app/node_modules/mitm/handlers/BaseHttpHandler.ts:61:23)
    at HttpRequestHandler.onRequest (/home/ec2-user/app/node_modules/mitm/handlers/HttpRequestHandler.ts:37:36)
    at Function.onRequest (/home/ec2-user/app/node_modules/mitm/handlers/HttpRequestHandler.ts:276:5)
------REMOTE CORE---------------------------------
    at Function.reviver (/home/ec2-user/app/node_modules/commons/TypeSerializer.ts:208:26)
    at JSON.parse (<anonymous>)
    at Function.parse (/home/ec2-user/app/node_modules/commons/TypeSerializer.ts:24:17)
    at WebSocket.<anonymous> (/home/ec2-user/app/node_modules/client/connections/RemoteConnectionToCore.ts:67:42)
    at WebSocket.emit (events.js:400:28)
    at WebSocket.emit (domain.js:475:12)
    at Receiver.receiverOnMessage (/home/ec2-user/app/node_modules/ws/lib/websocket.js:983:20)
    at Receiver.emit (events.js:400:28)
    at Receiver.emit (domain.js:475:12)
    at Receiver.dataMessage (/home/ec2-user/app/node_modules/ws/lib/receiver.js:517:14)
------CONNECTION----------------------------------
    at new Resolvable (/home/ec2-user/app/node_modules/commons/Resolvable.ts:17:18)
    at Object.createPromise (/home/ec2-user/app/node_modules/commons/utils.ts:68:10)
    at RemoteConnectionToCore.createPendingResult (/home/ec2-user/app/node_modules/client/connections/ConnectionToCore.ts:328:31)
    at RemoteConnectionToCore.internalSendRequestAndWait (/home/ec2-user/app/node_modules/client/connections/ConnectionToCore.ts:253:43)
    at RemoteConnectionToCore.sendRequest (/home/ec2-user/app/node_modules/client/connections/ConnectionToCore.ts:156:17)
    at Object.cb (/home/ec2-user/app/node_modules/client/lib/CoreCommandQueue.ts:104:26)
    at Queue.next (/home/ec2-user/app/node_modules/commons/Queue.ts:82:19)
------CORE COMMANDS-------------------------------
    at Queue.run (/home/ec2-user/app/node_modules/commons/Queue.ts:35:19)
    at CoreCommandQueue.run (/home/ec2-user/app/node_modules/client/lib/CoreCommandQueue.ts:100:8)
    at CoreTab.goto (/home/ec2-user/app/node_modules/client/lib/CoreTab.ts:92:36)
    at Tab.goto (/home/ec2-user/app/node_modules/client/lib/Tab.ts:160:36)
    at /home/ec2-user/app/src/scraper.js:245:32

--------------------------------------------------
------default-session-67--------------------------
------e2630310-39bb-11ec-88da-731dedebd20f--------
--------------------------------------------------

I was able to reproduce it by chance a single time in standalone testing.

The url happened to be this:

https://www.mragheb.com/Restoring%20The%20Global%20Equatorial%20Ocean%20Current%20Using%20Nuclear%20Excavation.pdf

@blakebyrnes
Copy link
Contributor

Are you running a custom browser plugin here? Looks like an http2 header is being sent to an http1 request. Can you send that session database? e2630310-39bb-11ec-88da-731dedebd20f.db

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants