Given an input file which is an attributes list of one place. The attributes of the place consists of a number of category instances. Each line represents one place. Examples are as following:

Breakfast & Brunch;American (Traditional);Restaurants
Sandwiches;Restaurants
Local Services;IT Services & Computer Repair
Restaurants;Italian
Food;Coffee & Tea
Fast Food;Restaurants
Mortgage Brokers;Home Services;Real Estate

So the task is to mine frequent category patterns under a given min-support.

The philosophy of Apriori algorithm includes two steps: frequent pattern generating and pruning. It’s an iterative process that starts from Candidates set and ends up with Frequent set. Starts from 1-item to 2-items and so on. For example,
Step 1, Scan data to generate 1-item candidate set. And filter the patterns with min-support to find frequent patterns.
Step 2, Generate 2-item candidate set based on 1-item set and
- Trim the data by removing infrequent 1-item patterns.
- Calculate support for 2-item patterns and prune to with min-support to get 2-item frequent patterns.
Step 3, repeat above steps 2 to generate 3-item frequent patterns and so on. Until there is no x-item patterns exist in data or no patterns satisfy the min-support.

So we’ll work on this “category of places” data to find all frequent patterns. What’s the value of this? To find what services are always together in one place could be used as references to open your new stores for best profits.
Let’s say if we find “Hotels & Travel”, “Hotels” and “Event Planning & Services” are associate most in some places what business chances you want to take there?

So we’ll get started.

It looks familiar to generate 1-item patterns calculating supports as to do a classic “word count”. So just use handy Spark cluster to get this done firstly.

After that we’ll need to adjust format and apply min-support (771) to get 1-item frequent set.

Here is a look at the 1-item frequent set - “support:category” as following:

Next, we’ll work on 2-item frequent set.
Step 1, we’ll generate 2-item candidate set based on 1-item frequent set using itertools.combinations method.

Step 2, we’ll remove infrequent 1-item categories from dataset and then calculate support for 2-item candidate category set.

Now we have 2-item frequent category set.

So we can continue to get 3-items frequent category set like this - only one line:

By now we can also tell this is the end of the frequent patterns - three items cannot make up four or more patterns.

Lastly we can combine all the frequent patterns: 1-item, 2-item and 3-item patterns together to get all the frequent patterns out of this dataset.