coolguy February 2016

Logging in and redirecting into a website using jsoup

Iam presently using jsoup library for parsing websites.Can I use jsoup.connect(url).get() using login info. user id and password or else any other library that could provide this function.I have seen some solutions using login page parsing of the website.But is there any way to directly connect using user name and pwd.Thanks for help.

Answers


Alexey Gromov February 2016

Short answer: it depends.

You have to understand how the user authenticates itself on the site you are parsing. May be the username and password are sent as GET/POST request parameter (if the login form is used, see "Form-based authentication") or the site uses web container authentification, that is configured in the web.xml (Basic, NTLM, Kerberos etc.).

In case of form-based authentication accepting POST requests you can use

Jsoup.connect(url).data("user","username","pwd","1234").post();

if your login form has an input named user for username and pwd for password. Be aware that you will send the password unencrypted over the network (unless you use https).

For other authentifications consider using a full-sized HTTP client (e.g. Apache HttpClient) and RTFM (see Credentials Provider, User Credentials).

You will then one of Jsoup.parse(...) Methods to parse the responses you will get using the HTTP Client of your choice.

Post Status

Asked in February 2016
Viewed 2,527 times
Voted 4
Answered 1 times

Search




Leave an answer