finalprogramme: (0)
K. ([personal profile] finalprogramme) wrote in [site community profile] dw_dev 2023-06-16 05:36 pm (UTC)

Novice barely-knows-how-to-code question here.

I'm trying to scrape comments from a community with an HTTPS call, using the URL:

https://www.dreamwidth.org/export_comments?get=comment_body&startid=0&authas=COMMNAME

This works if I'm logged in to the browser, no problem. But what I'm struggling with is how to send that request as part of a script. So for example, I put together what I thought would be a simple Python script:

import requests

from requests.auth import HTTPBasicAuth

auth=HTTPBasicAuth('USERNAME', 'APIKEY')

r = requests.get('https://www.dreamwidth.org/export_comments?get=comment_body&startid=0&authas=COMMNAME', auth=auth) print(r.text)

What I get back is the Dreamwidth login page. I'm sure there's something extremely obvious that I'm missing here, but dammit Jim, I'm a docs writer, not a developer.


Post a comment in response:

If you don't have an account you can create one now.
HTML doesn't work in the subject.
More info about formatting

If you are unable to use this captcha for any reason, please contact us by email at support@dreamwidth.org