<div style="font-family: Arial, sans-serif; font-size: 14px;"><span style="font-family: Menlo; font-size: 11px;">Hi all,</span></div><div style=""><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">We were having an issue with the reverse proxy on any requests where the uri had anything after the base (e.g. servpub.net/anythingelse), these requests weren't being passed on correctly by Jean reverse proxy to our other nginx servers on the raspberry pis. The issue was affecting a media wiki site (using php) and also was affecting the serving of all static files (e.g. images).</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-family: Menlo; font-size: 11px; color: black;">I did some analysis of the nginx logs on Jean and the pi, and Mara was able to check / confirm that the solution below works. </span><span style="font-family: Menlo; font-size: 11px; color: black;">So I'm sharing the original analysis and resource where the solution was found as it's maybe useful to have the solution and the strategy used for figuring out/confirming what the issue was.</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="margin: 0px; min-height: 13px; line-height: normal; background-color: rgb(255, 255, 255);"><font face="Menlo"><span style="font-size: 11px;">Best,</span></font></p><p style="margin: 0px; min-height: 13px; line-height: normal; background-color: rgb(255, 255, 255);"><font face="Menlo"><span style="font-size: 11px;">Becky / In-grid</span></font></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">---</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">If you go to wiki2print.servpub.net, the request access log on Jean will show:</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">[15/Dec/2023:14:49:40 +0100] "GET / HTTP/1.1" 301 5 </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">Request gets immediately passed to wiki2print pi:</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">10.10.12.2 - - [15/Dec/2023:13:49:40 +0000] "GET / HTTP/1.0" 301 0 </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">The 301 is showing that the php is doing what is expected by redirecting to the “/index.php?title=Main_Page” location, we then see this immediately in Jean access log:</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">[15/Dec/2023:14:49:40 +0100] "GET /index.php?title=Main_Page HTTP/1.1" 404 188 </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">That request gives the 404 so it does not get passed to wiki2print (no record in nginx error or access log on wiki2print).</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">So I thought that what was happening with the above requests resulting in a 404 was the reverse proxy on Jean not handling the <span style="display: inline !important; background-color: rgb(255, 255, 255);">“/index.php?title=Main_Page”</span> request correctly, the same issue happening with the serving of static files. So based on that I found this forum post which indicates that try_files is the thing that was causing our issue.</span></p><div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; margin: 0px; color: black; background-color: rgb(255, 255, 255);"><br></div><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><a href="https://serverfault.com/questions/780585/nginx-403-when-serving-static-files-reverse-proxy" target="_blank" style="margin:0px">https://serverfault.com/questions/780585/nginx-403-when-serving-static-files-reverse-proxy</a> </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">Solution based on the forum post above:</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">location @proxy {</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> proxy_pass<span> </span><a href="http://10.10.12.4/" target="_blank" style="margin:0px">http://10.10.12.4</a>;</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">}</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">location / {</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> #other proxy stuff already there</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> # First attempt to serve request as file, then</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> # as directory, then fall back to displaying a 404.</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> # try_files $uri $uri/ =404; # <-- this is the problem</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> # simply pass the uri to the proxy to be handled by the server on the pis</span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"> try_files $uri @proxy; </span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; min-height: 13px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;"><br></span></p><p style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, "system-ui", Roboto, "Helvetica Neue", sans-serif; font-size: 15px; margin: 0px; line-height: normal; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);"><span style="font-size: 11px; font-family: Menlo; margin: 0px; color: black;">}</span></p><br><br></div><div style="font-family: Arial, sans-serif; font-size: 14px;"><br></div>
<div class="protonmail_signature_block" style="font-family: Arial, sans-serif; font-size: 14px;">
<div class="protonmail_signature_block-user protonmail_signature_block-empty">
</div>
<div class="protonmail_signature_block-proton">
Sent with <a target="_blank" href="https://proton.me/" rel="noopener noreferrer">Proton Mail</a> secure email.
</div>
</div>