I need to download a user list from a secured website I have access to. Unfortunately, I can only view 25 results at a time. So there are 294 pages of results to go through before I can see all the users. Rather than open each page one-by-one and download it's source, I'd like to use an Automator workflow.
I tried making a webpage with links to all the result pages I'd need (while having a Safari window open that was logged in), then inputting that page into Get Specific URLs, Get Link URLs from Webpages to get those 294 URLs I'd need, Get Text from Webpage, and finally, New Text File. Unfortunately, that text file is just error message after error message. The website was treating Automator's requests as if it hadn't logged in.
So I have two questions:
I tried making a webpage with links to all the result pages I'd need (while having a Safari window open that was logged in), then inputting that page into Get Specific URLs, Get Link URLs from Webpages to get those 294 URLs I'd need, Get Text from Webpage, and finally, New Text File. Unfortunately, that text file is just error message after error message. The website was treating Automator's requests as if it hadn't logged in.
So I have two questions:
- Can Automator log in or act like it's logged in, since Safari is?
- If not, could I make a workflow that would get those 294 URLs, open one in a Safari window, download the source, close the window, and repeat for the next URL? I don't want 294 windows open at once, I don't think my iBook could take it.