For a gentle entry into parallel programming, we’ll process and analyze a real-world weather buoy dataset. This is a miniature example of the big data challenge: how to process a large amount of data that may be too big to fit into memory, or may take too long to process sequentially. The crux of the problem is in the sheer volume of data; however, the computation itself can typically be carried out on subsets of the data independently from one another. For this exercise to be tenable and easily reproduced on personal computers, I created a sample of a real weather dataset that’s small enough to be easily downloaded, but large enough to demonstrate the application of parallel data processing with Fortran.
In this exercise, you’ll learn the following:
- How to distribute the data and workload between parallel processes
- How to exchange intermediate results between parallel processes using Fortran coarrays
I’ll first describe the data and show you how to get it, and then we’ll dive straight into the serial implementation of the program. We’ll then identify which parts of the program we can parallelize and how.