Configuring IPython Notebook Support for PySpark by John Ramey.
From the post:
Apache Spark is a great way for performing large-scale data processing. Lately, I have begun working with PySpark, a way of interfacing with Spark through Python. After a discussion with a coworker, we were curious whether PySpark could run from within an IPython Notebook. It turns out that this is fairly straightforward by setting up an IPython profile.
…
A quick setup note for a useful configuration of PySpark, IPython Notebook.
Good example of it being unnecessary to solve every problem to make a useful contribution.
Enjoy!