Lessons
Start
Start learning
CodeGym /Courses /Java Collections /Big task: Java aggregator

Big task: Java aggregator

Java Collections
Level 8 , Lesson 15
Available

"Hello, soldier!"

"Congratulations on upgrading your skills. We need guys who are prepared to do anything."

Java university

"I'm sure you still have many unfinished tasks. It's time to finish some of them!"

10
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 1)
It's time to work with information from the Internet a little. In this task, you'll be writing an aggregator for Java job posting. What do we need? There should be a list of websites we are searching for jobs. First, we'll use https://www.linkedin.com/jobs/. We'll add other job search sites later.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 2)
1. Create a Controller class which will contain business logic. 2. Add a public constructor to Controller. It should accept as many providers as are passed to it. Save them to a private providers field. Do you remember how to do that?
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 3)
Starting in this task, you will begin writing the logic for fetching data from a website. This logic will be contained entirely in the classes that implement Strategy. Speaking in terms of the Strategy pattern, the provider acts as the context.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 4)
Open the Linkedin website (https://www.linkedin.com/jobs/) Type "Java San Francisco" in the search bar. At the bottom, navigate to the second page, since URLs often differ between the first and subsequent pages.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 5)
1. Add a getJobPostings(String searchString) method to the interface. It will return a list of job postings. 2. Correct the errors in the LinkedinStrategy class. 3. Return to the Provider class's getJavaJobPostings method. Implement it assuming that all the data is adequate.
Open
36
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 6)
1. On the left in the Project panel, find "External Libraries" at the bottom. Find the examples package in jsoup, and look at the classes in this package. 2. Similar to the implementations in the jsoup examples, connect to the Linkedin URL using the GET method.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 7)
To let Linkedin know who's connecting to it, we'll add Request Headers to our request. There are developer tools that show various information about requests. I'll tell you about two tools. ***Chrome**** 1. In the Chrome browser, go to Menu -> Tools -> Developer Tools, or press Ctrl+Shift+I.
Open
36
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 8)
Run the program in debug mode again. Copy the value of document.html() into the HTML file we created earlier. Format it and find the tags with job postings. Read additional lesson material about attribute selectors in the Community. PLEASE NOTE THIS SPECIAL RULE FOR TESTING THIS TASK!
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 9)
It's time to refactor the code a little. Read about the Model-View-Controller (MVC) pattern on the Internet. In short, the user uses the view to generate events that are processed by the controller. The controller decides what data it needs, and accesses the correct model.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 10)
View has an update method. The list of job postings is passed to it to be displayed. Obviously, only the model will call this method, since only the model gets the data. It's time to create the model. 1. Create a Model class in the model package.
Open
10
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 11)
Model has a selectCity method to which the name of the city for the job search is passed. Obviously, this method will be called by the controller, since it decides which model to use. 1. Add a new Model model field to the Controller.
Open
10
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 12)
To run the application, you need to emulate a user event: 1. In the HtmlView class, create a public void emulateCitySelection() method. Let this method call the controller using the city "Odessa".
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 13)
Check it out — there are two new files in the view package: jobPostings.html (you will write data here), backup.html (a copy of jobPostings.html for data recovery if the data in jobPostings.html is erased somehow). Hover over and right-click jobPostings.html in the project tree on the left.
Open
18
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 14)
There are still two empty methods in the HtmlView class. In this task, I'll describe what the updateFile method should do. And I'll also tell you how to debug it. It accepts the file's body as a String. You need to write it to the file located at the filePath path.
Open
36
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 15)
In the HtmlView class, only the getUpdatedFileContents method is still empty. In this task, I'll describe what it should do. 1. In HtmlView, create a protected Document getDocument() throws IOException method that parses the jobPostings.html file using jsoup. The file's encoding is UTF-8.
Open
36
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 16)
You are doing great! You've done a lot of work! Now it will be easy for you to monitor job vacancies :) Aggregator currently uses only one strategy for collecting job postings (from Linkedin). 1. Add a strategy for Indeed by analogy with LinkedinStrategy.
Open
10
Task
Java Collections, level 8, lesson 15
Locked
Aggregator (part 17)
Stuff you can do on your own (we won't test you on any of these items): 1. Add a bazillion other job sites for aggregating job postings. You just need to create a strategy, and then add a provider for that strategy to the model. 2. Sort all of the job postings.
Open
Comments (12)
TO VIEW ALL COMMENTS OR TO MAKE A COMMENT,
GO TO FULL VERSION
Hoist Level 4, San Diego, United States
6 March 2025
Think about this (partial) module here --- most of the work in here Strategy is an interface type that's being used in the Provider class. The Provider class has: A private field named strategy that is of type Strategy (interface) A constructor that takes a Strategy parameter to initialize this field A setter method that allows changing the strategy field after the object is created This implementation follows the Strategy design pattern, where: Provider is the context class that uses a strategy Strategy is the interface that defines the contract for all concrete strategy implementations --- The concrete strategy classes would implement the Strategy interface // add the class up here private Strategy strategy; public Provider(Strategy strategy) { this.strategy = strategy; } public void setStrategy(Strategy strategy) { this.strategy = strategy; } }
Ibrahim Level 41, Sheffield, United Kingdom
27 June 2022
That wasn't too bad a task just wish there was more explanation on how to use Jsoup.
Szymon Level 41, Warsaw, Poland
30 December 2021
https://hexometer.com/http-status-codes/ Read about error no. 999: "Status code 999 - a non-standard code is returned by some sites (e.g. LinkedIn) which do not permit scanning." Doesn't it mean that creating such a program is basically blocked by LinkedIn itself?
Thành Black Level 49, Hanoi
17 December 2021
Install Jsoup with Maven: 

<!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.14.3</version>
</dependency>
Andrei Level 41
28 July 2021
How do I add the jsoup library? It only has .java files when I download it...
Dave Andrea Level 41, Canada
2 October 2020
Well... that was a mess. The instructions were actually wrong in multiple places and the validator was all over the place. No wonder only 24 people completed this. It was actually a pretty cool task to implement, but it was way more painful than it needed to be. If anyone needs help on this one you can message me. or message me the link to your question in the help section.
Juan Ma Level 41, Arauca, Colombia
27 June 2020
If you are not acquainted with html I recommend you this course you will need it. In tasks 8 and 16 just add a break before going to the next page since the codegym cache web only have one page. Happy coding :)
JianQiu Hwang Level 35, Washington
16 May 2020
How incomprehensible! It should be rewrited.
fzw Level 41, West University Place, United States
12 May 2020
Validation is a little bit dumb. For Linkedin strategy, remove duplicate job postings. But don't remove duplication for Indeed strategy Indeed strategy: private static final ... URL_FORMAT will not pass validation. use "static private final ..."
Senned Level 41, Azov, Russia
10 January 2020
Aggregator (part 8) The average number of attempts for this task is 77.0. O_O
Senned Level 41, Azov, Russia
10 January 2020
19 attempts becouse in conditions not consist that List of jobPosting must have unique values