Skip to content
Snippets Groups Projects
Commit 028a1ec1 authored by Zhichao's avatar Zhichao
Browse files

add hint

parent aa4e725d
Branches master
No related tags found
No related merge requests found
......@@ -118,7 +118,7 @@ The format of the log entries should be self-explanatory, but here are more deta
Note: In this question, both list of ratings and users can be accepted as the values in the answer RDD(the result.txt is taking users as the value). We highly recommend you to implement the solution with list of users, since the input of Task5 function is a tuple of (user, product), which means you don't need to make any change to assignment.py
- **Task 6 (4pt)**: On the logsRDD, for two given times, use a 'cogroup' to create the following RDD: the key of the RDD will be a host, and the value will be a 2-tuple, where the first element is a list of all URLs fetched from that host before the first time, and the second element is the list of all URLs fetched from that host after the second time. Use filter to first create two RDDs from the input logsRDD.
- **Task 6 (4pt)**: On the logsRDD, for two given times, use a 'cogroup'(function in Spark) to create the following RDD: the key of the RDD will be a host, and the value will be a 2-tuple, where the first element is a list of all URLs fetched from that host before the first time, and the second element is the list of all URLs fetched from that host after the second time. Use filter to first create two RDDs from the input logsRDD.
- **Task 7 (8pt)**: Your task is to write a name counting application for counting the first names of the Nobel Prizes for each category. The return value should be a PairRDD where the key is a string, of which the format is "[Category]:[Firstname]" (i.e., "chemistry:Michael"), and the value is its count, i.e., in how many times did that combination appear.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment