add userguide to docklet-book

This commit is contained in:
Donggang Cao 2016-03-14 18:43:44 +08:00
commit ad09f286e8
111 changed files with 9110 additions and 0 deletions

2
.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
*~
.DS_Store

4
README.md Normal file
View File

@ -0,0 +1,4 @@
Docklet Document
================

6
userguide/.gitignore vendored Normal file
View File

@ -0,0 +1,6 @@
_book
*~
.DS_Store
node_modules
.epub
book*pdf

2
userguide/LANGS.md Normal file
View File

@ -0,0 +1,2 @@
* [中文](zh)
* [English](en)

30
userguide/Makefile Normal file
View File

@ -0,0 +1,30 @@
# generating html site using gitbook
# uses language-picker plugin, so need to install it
#
# npm install gitbook-plugin-language-picker
#
html: check
gitbook build
pdf: check
gitbook pdf
epub: check
gitbook epub
check:
@if [ -n "`which gitbook`" ] ; then \
echo "gitbook found" ; \
else \
echo "gitbook not found, try: npm install gitbook-cli -g" ; \
exit 1 ; \
fi
serve:
gitbook serve
all: html pdf epub
clean:
rm -rf book*.pdf *.epub _book node_modules

5
userguide/README.md Normal file
View File

@ -0,0 +1,5 @@
Docklet User Guide
==================

5
userguide/book.json Normal file
View File

@ -0,0 +1,5 @@
{
"plugins": [
"language-picker"
]
}

BIN
userguide/cover.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 151 KiB

BIN
userguide/cover_small.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

10
userguide/en/README.md Normal file
View File

@ -0,0 +1,10 @@
Docklet User Guide
===============
v0.2
Feb 20, 2016
© Software Engineering Institute, Peking University
http://www.unias.org/trac/docklet

37
userguide/en/SUMMARY.md Normal file
View File

@ -0,0 +1,37 @@
# Summary
* [Docklet Introduction](intro/README.md)
* [What is Docklet](intro/what.md)
* [Why to use Docklet](intro/why.md)
* [How to use Docklet](intro/how.md)
* [Basic Concepts](concepts/README.md)
* [Workspace](concepts/workspace.md)
* [vCluster](concepts/vcluster.md)
* [Image](concepts/image.md)
* [Service](concepts/service.md)
* [Dataspace](concepts/nfs.md)
* [Portal](portal/README.md)
* [Dashboard](portal/dashboard.md)
* [Config](portal/config.md)
* [Status](portal/status.md)
* [Hosts](portal/hosts.md)
* [Users](portal/users.md)
* [Admin](portal/admin.md)
* [Workspace](workspace/README.md)
* [Data Management](workspace/data.md)
* [File Editing](workspace/edit.md)
* [WEB Terminal](workspace/terminal.md)
* [Interacive Programming](workspace/notebook.md)
* [Development in WEB Terminal](software/README.md)
* [Available Tools](software/list.md)
* [C/C++](software/c.md)
* [Java](software/java.md)
* [MPI](software/mpi.md)
* [Spark](software/spark.md)
* [LaTeX](software/latex.md)
* [Interactive Computing and Analyzing](notebook/README.md)
* [Python Packages](notebook/python.md)
* [R Packages](notebook/r.md)
* [Notebook Usage](notebook/nb.md)
* [Scientific Computation](notebook/gallery.md)

View File

@ -0,0 +1,9 @@
# Basic Concepts #
Docklet has the following basic concepts:
* [Workspace](workspace.md)
* [Vcluster](vcluster.md)
* [Image](image.md)
* [Service](service.md)
* [Dataspace](nfs.md)

View File

@ -0,0 +1,24 @@
## Image ##
The concept of image in Docklet is similar to that in [Docker](https://github.com/docker/docker), i.e., a readonly template to create containers. But there are some differences.
Since Docker mainly focuses on LXC container in IaaS level, its images are
very free and versatile. Users can customize and make their own images
and then upload them to the docker hub. Usually one Docker image has
only the environment of one specific application. For example, a Docker
image may have a full Ubuntu operating environment, in which there is
only the Apache web server software.
Docklet is a bit different. Docklet focuses on **Workspace** and
**vcluster**, therefore has special restrictions about images, not
supporting uploading user-made images. The Docklet administrator will
make one or more public base image according to the common requirements
of the enterprise, containing the needed software packages to support
Workspace, together will some programming tools and frameworks such as
MPI, Spark, Python and R. Since the Docklet public base images are
shared by all users, therefore is usually larger than typical Docker
containers.
If users can not find needed software tools in the public base image,
they can install them in their Workspace. They can then save their
Workspace as private image, and can choose to share it with others.

View File

@ -0,0 +1,19 @@
## Dataspace ##
In Docklet, each user has their own persistent Dataspace, in which to
save data files. Dataspaces are isolated from each other, each user can
only visit their own Dataspace.
User's Dataspace is shared among all nodes of their vcluster. It is
located in the `~/nfs` directory of the container node.
When the user save their Workspace as image, the Dataspace will be
excluded.
The Dataspace is supported by distributed file system in the backend.
Except the Dataspace, all data in containers will be destroyed as
the containers are deleted. But the data in Dataspace will be persistent
regardless of the lifecycle of containers.
Users are encouraged to download and backup their important data in
time.

View File

@ -0,0 +1,10 @@
## Service ##
Many sofware tools in Docklet image can be run directly as script in the
terminal. But there are also some software frameworks need to run as
service, e.g., [Spark](http://spark.apache.org). When users create a
Workspace, those frameworks are not started in default, to reduce
resource consumption. Users may configure their Docklet vcluser, to
allow starting specified software frameworks.
Docklet can be configured to start user-installed software services.

View File

@ -0,0 +1,26 @@
## vcluster ##
Vcluster(virtual cluster) is the basic concept in Docklet. A vcluster
consists of at least one LXC container nodes, each node can be regarded
as a lightweight Linux host. Usually, the container nodes of one
vcluster are generated from the same image, which can simplify the
management of vcluster.
The level of the user determines the quota of his vcluster, including
the maximum number of nodes the vcluster can have, the CPU and memory
config of each container node, etc. The default vcluster has one node.
The nodes of Docklet vcluster are designed to distribute to different
physical hosts as most as possible, in order to better utilize physical
resources. Those distributed nodes can communicate with each other. One
docklet vcluster will use an independent virtual sub-network, usually
the first node is the gateway of this sub-network.
In most cases, users do not need to care about the details of vcluster,
but only need to focus on their Workspace.
In some special cases, users may need to configure the vcluster to meet
the requirements of upper Workspace. For example, in order to run
program parallelly in Workspace, users need to scale out the default
vcluster, adding more nodes. Docklet vcluster has the ability of elastic
scaling, supporting adding or removing nodes at any time.

View File

@ -0,0 +1,23 @@
## Workspace ##
Workspace is the core concept in Docklet. It represents a WEB based
interactive working environment. The Docklet Workspace is provided by an
open source component [Jupyter Notebook](https://jupyter.org), by which
users can create and share documents that contain live code, equations,
visualizations and explanatory text. It can be used in the following
area: data cleaning and transformation, numerical simulation,
statistical modeling, machine learning and much more.
For most users, they do not care about the low level details of physical
hosts, clusters, containers or vclusters. They only need to open a
browser to enter their personal Workspace, then interactively do
their jobs, even including visualizations in the browser. Currently the
Jupyter Notebook supports more than 40 programming languages.
The following is a snapshop of [Jupyter Notebook](https://jupyter.org) Workspace:
<img src="../images/jupyterpreview.png" width="600" alt="jupyter workspace">
For advanced users, Jupyter Notebook offers WEB Terminal to accomplish
all most all non-visualization works.

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 322 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

View File

@ -0,0 +1,14 @@
# Docklet Introduction #
Welome to the world of Docklet!
What is Docklet?
Does it have any relationship with the famous Docker project?
What advantages it has?
How to use Docklet?
This section will answer those questions for you first.

14
userguide/en/intro/how.md Normal file
View File

@ -0,0 +1,14 @@
## How to use Docklet ##
For Docklet users to access their Workspace, what they need is only a
modern web browser, such as Firefox, Chrome or Safari.
Docklet administrator will configure the authentication method.
For example, the instance deployed at the Computing Center in Peking
University, http://iwork.internetware.org , supports authenticating both
Peking University accounts and Docklet internal accounts.
Docklet also supports Guest account. Users can try the "Workspace" using
the guest account with some limitations. The Guest Workspace will
restart periodically.

View File

@ -0,0 +1,53 @@
## What is Docklet ##
### Definition and Architecture ###
Docklet is an open source project from Software Engineering Institute,
Peking University. It is released uner the [New BSD License](http://directory.fsf.org/wiki/License:BSD_3Clause). Its source code is hosted in GitHub.
The goal of Docklet is to provide a solution of **Personal Development
Workspace in the Cloud** for enterprises. With the help of Docklet,
enterprises can easily virtualize their small-scale data centers,
creating virtualized clusters for their staff users, and then providing
users a customizable **Workspace** in the cloud. Users only need a
modern **browser** to visit their own **Workspace** located in the
enterprise's Intranet from anywhere, at anytime. They can do works like
online editing source codes, debugging, testing, managing data files,
analyzing data, visualizing results, etc.
The kernel technology of Docklet is LXC virtual cluster (vcluster) that
is based on Linux Container (LXC) and software defined networks
technologies. The Docklet **vcluster** provides a virtualization
solution of physical clusters, supports the installation and running of
most existing Linux applications.
Based on vcluster, Docklet provides users their **Personal Workspace**.
Users visit their Workspace throught the integrated [Jupyper Notebook](https://github.com/jupyter/notebook) to do most development tasks.
The architecture of Docklet is illustrated as follows:
<img src="../images/docklet-architecture.jpg" width="500"
alt="docklet-architecture">
<!--
![docklet architecture](../images/docklet-architecture.jpg)
-->
### Docklet and Docker ###
Docklet has no direct relationship with [Docker](https://github.com/docker/docker). They are two different projects, with totally different design goals.
The goal of Docker is to provide a lightweight operating system
virualization solution. It is in the [IaaS layer](https://en.wikipedia.org/wiki/Cloud_computing) of cloud computing architecture.
Docker encapsulates LXC to make LXC easy,
so that [the users can operate a Docker container as simple as operate
a fast and lightweight virtual machine](https://www.gitbook.com/book/yeasy/docker_practice). Although later on Docker has clustering component,
it in essence still focuses on single node operating system layer.
The goal of Docklet is to provide **Personal Development Workspace in
the Cloud** solution. It covers all the [SaaS, PaaS and IaaS layer](https://en.wikipedia.org/wiki/Cloud_computing) of cloud computing architecture.
The basic of Docklet is LXC vcluster, but not Docker container. For
Docklet users, what they face directly is their Workspace. They use
browser to do software development, debuging and testing, etc, using tools
Docklet provide, working in a high layer.

104
userguide/en/intro/why.md Normal file
View File

@ -0,0 +1,104 @@
## Why to use Docklet ##
As a "**Workspace Cloud**", Docklet has several advantages.
First, what Docklet provides is a virtual workspace for developers, but
no only an Iaas or PaaS solution, therefore more friendly to deveopers.
Second, the Docklet vcluster provides a high effient and simple method
of sharing physical resources, therefore more friendly to
administrators.
The detailed advantages and features of Docklet include:
### Wide applicability ###
Docklet is not designed for large data center, but for small or **mini
data center**. By **mini data center**, we mean a cluster with not more than 100
physical hosts, all connected by high speed network, each host with
relatively high performance. This is the common configuration of most
medium or small enterprises and institutes. Therefore, Docklet has a
wide applicability.
### Easily Sharing Cluster among Frameworks ###
Traditionally clusters are often monopolized, serving only one kind of
application framework, e.g., MPI or Spark. It is usually difficult to
share the physcal cluster among different frameworks.
[Mesos](http://mesos.apache.org) and
[Yarn](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html) each try to provide a mechanism of sharing clusters. However, they require extra work of customization for each supported framework, which seems not flexible and a bit difficult.
Docklet provides a flexible and easy solution. By virtualizing the
physical clusters, almost all computing frameworks can run in the
LXC vcluster seamlessly without any customization and modification.
### Dynamically and Elastically Configure Resources ###
Docklet vcluster supports elastically configuring resources. Users can
add a fully functional LXC node to existing vcluster instantly by just a
click. This feature is useful for the application frameworks that
support elastic resource management, e.g., Spark.
Docklet can monitor the activity of users, smartly finding those not at work,
then adjust their resource quota dynamically. This can bring about good
usage experience for online active users, together with reasonable
resource utilization.
### Consuming Very Few Network Resources ###
Docklet utilizes the technology of software defined networks.
The inner vclusters will use private Internet address space, e.g.,
`172.16/16`, not consuming public Internet addresses. The administrator
only need to configure **one** public Internet address/URL for Docklet
portal. Users use just a modern browser to access their private
vcluster located behind the enterprise's firewall through the portal's
public URL. This feature is very attractive for those enterprises
without so many public Internet addresses.
### User Friendly ###
Docklet is mainly targeted for enterprises's staff developers. It is
domain oriented, therefore can be setup to meet the enterprise's
requirements deliberately. The public base image has most often used
software. Users can directly start their programming and analyzing job
with just a browser, without the heavy burden of installing and
configuring the physical clusters, installation of software frameworks,
management of images, etc. Therefore, it is very friently to end users,
which is especially important for those non-export users.
### Ideal Environment for Application Development and Testing ###
The easy usage of Docklet means it can be an ideal environment for
application development and testing. For example, users can program and
debug a big data analyzing software with a small data set (possibly
Gigabye scale). If success, then deploy it to production system with
true big data. In fact, for most data analyzing and mining jobs, their
data set is usually not so large, which means they can directly run and
output the final results in Docklet.
Docklet is also a good platform for exercising programming skills and
studying programming languages. No need to install kinds of tools, just
a browser is OK.
### Easy to Customize and Extend ###
The enterprise's Docklet administrator is free to make specific public base
images. The users can also install new software not in the base image.
What is more, the users can save their Workspace as image and then share
with others, which is very convenient for users of one group.
### Supporting Multiple Users ###
Docklet vcluster naturally supports multiple users. Each user has their
own private vcluster independenlty. Each vcluster can run different
frameworks such as MPI and Spark without worry about interference.
Therefore, it can satisfy the different requirements of different users.
### Advantages of LXC ###
Docklet uses LXC technology. Therefore, it has the advantages of LXC and
LXC-based solutions like Docker, including instant booting container in seconds;
high utilization of resources, running thousands of containers in one
physical host; high performance of applications; consuming less
resources; fast delivery; simple management, etc.

View File

@ -0,0 +1,39 @@
# Scientific Computation and Data Analysis #
Docklet supports scientific computation and data analysis mainly by two languages [Python](https://www.python.org) and [R](http://www.r-project.org), as well as plentiful packages.
Most work can be done in Jupyter Notebook, including data visualization,
with only a browser.
There are many articles in the WEB about how to use Pyton and R do big
data processing, scientific computing, machine learning and deep
learning, natural language processing, statistics, data mining, graph
processing, data visualization, etc. In these articles, what is valuable
to novice users is what packages can be used in what kind of tasks.
[ANACONDA](https://www.continuum.io/why-anaconda) from
[CONTINUUM](https://www.continuum.io) is a computation and analysis
platform. It integrates both useful open source Python packages and R packages.
The [Canopy](https://www.enthought.com/products/canopy/) from
[ENTHOUGHT](https://www.enthought.com) is a platform similar to CONTINUUM.
It also integrates many useful Python packages for computation and
analysis.
[RStudio](https://www.rstudio.com) introduces [RStudio
IDE](https://www.rstudio.com/products/rstudio2/) and several
[useful R packages](https://www.rstudio.com/products/rpackages/).
Though many visualization and IDE packages need to run in local
OS, most packages in Docklet can run in Jupyter Notebook as Web app.
Users can use `pip` or other tools to install those packages needed.
The list of Python and R packages in Docklet:
* [Python Packages](python.md)
* [R Packages](r.md)
About the tutorials and examples about using Jupyter Notebook in
activities like scientific computation, please referer
[nbview](http://nbviewer.jupyter.og) and [ipython
wiki](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks).

View File

@ -0,0 +1,588 @@
## Gallery of Some Interesting Notebooks##
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks shows a gallery of interesting notebooks in various situations.
Some notebooks require specific packages. Docklet has already preinstalled many
ofen-used [Python3](python.md) and
[R](r.md) packages. If the users find some packages not in Docklet, please install them manually, through `pip3`, R shell, etc.
1. [Entire books or other large collections of notebooks on a topic](#entire-books-or-other-large-collections-of-notebooks-on-a-topic)
* [Introductory Tutorials](#introductory-tutorials)
* [Programming and Computer Science](#programming-and-computer-science)
* [Statistics, Machine Learning and Data Science](#statistics-machine-learning-and-data-science)
* [Mathematics, Physics, Chemistry, Biology](#mathematics-physics-chemistry-biology)
* [Earth Science and Geo-Spatial data](#earth-science-and-geo-spatial-data)
* [Linguistics and Text Mining](#linguistics-and-text-mining)
* [Signal Processing](#signal-processing)
* [Engineering Education](#engineering-education)
1. [Scientific computing and data analysis with the SciPy Stack](#scientific-computing-and-data-analysis-with-the-scipy-stack)
* [General topics in scientific computing](#general-topics-in-scientific-computing)
* [Social data](#social-data)
* [Psychology and Neuroscience](#psychology-and-neuroscience)
* [Machine Learning, Statistics and Probability](#machine-learning-statistics-and-probability)
* [Physics, Chemistry and Biology](#physics-chemistry-and-biology)
* [Economics and Finance](#economics-and-finance)
* [Earth science and geo-spatial data](#earth-science-and-geo-spatial-data)
* [Data visualization and plotting](#data-visualization-and-plotting)
* [Mathematics](#mathematics)
* [Signal and Sound Processing](#signal-and-sound-processing)
* [Natural Language Processing](#natural-language-processing)
* [Pandas for data analysis](#pandas-for-data-analysis)
1. [General Python Programming](#general-python-programming)
1. [Notebooks in languages other than Python](#notebooks-in-languages-other-than-python)
* [Julia](#julia)
* [Haskell](#haskell)
* [Ruby](#ruby)
* [Perl](#perl)
1. [Miscellaneous topics about doing various things *with* the Notebook itself](#miscellaneous-topics-about-doing-various-things-with-the-notebook-itself)
1. [Reproducible academic publications](#reproducible-academic-publications)
1. [Other publications using the Notebook](#other-publications-and-conference-abstracts-that-explicitly-use-the-notebook)
1. [Data-driven journalism](#data-driven-journalism)
1. [Whimsical notebooks](#whimsical-notebooks)
1. [Videos of IPython being used in the wild](#videos-of-ipython-being-used-in-the-wild)
## Entire books or other large collections of notebooks on a topic
### Introductory Tutorials
* First things first, how to [run code in the notebook](https://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Running%20Code.ipynb). There is also a general [collection of notebooks](https://nbviewer.jupyter.org/github/ipython/ipython/blob/master/examples/IPython%20Kernel/Index.ipynb) from IPython. Another useful one from this collection is an explanation of our [rich display system](https://nbviewer.jupyter.org/github/ipython/ipython/blob/master/examples/IPython%20Kernel/Custom%20Display%20Logic.ipynb).
* A [great matplotlib tutorial](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-4-Matplotlib.ipynb), part of the fantastic [Lectures on Scientific Computing with Python](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/tree/master) by [J.R. Johansson](https://github.com/jrjohansson).
* The code of the [IPython mini-book](https://github.com/rossant/ipython-minibook) by C. Rossant, introducing IPython, NumPy, SciPy, Pandas and matplotlib for interactive computing and data visualization.
* [Python Tutorial](https://github.com/rajathkumarmp/Python-Lectures) by [Rajath Kumar M P](https://github.com/rajathkumarmp)
### Programming and Computer Science
* [Introduction to Programming (using Python)](http://nbviewer.ipython.org/github/ehmatthes/intro_programming/blob/master/notebooks/index.ipynb), an entire introductory Python course written by [Eric Matthes](http://peak5390.wordpress.com/about). [This post](http://peak5390.wordpress.com/2013/09/22/how-ipython-notebook-and-github-have-changed-the-way-i-teach-python/) explains the educational context in an Alaskan high school where Eric is a teacher.
* [Python for Developers](http://ricardoduarte.github.io/python-for-developers), a complete book on Python programming by [Ricardo Duarte](https://github.com/ricardoduarte). Note the book also exists [in Portuguese](http://ricardoduarte.github.io/python-para-desenvolvedores).
* [CS1001.py - Extended Introduction to Computer Science](https://github.com/yoavram/CS1001.py). Recitations from Tel-Aviv University introductory course to computer science, assembled as IPython notebooks by [Yoav Ram](http://www.yoavram.com/).
* [Exploratory Computing with Python](http://mbakker7.github.io/exploratory_computing_with_python/), a set of 15 Notebooks that cover exploratory computing, data analysis, and visualization. No prior programming knowledge required. Each Notebook includes a number of exercises (with answers) that should take less than 4 hours to complete. Developed by Mark Bakker for undergraduate engineering students at the Delft University of Technology.
* [Understanding evolutionary strategies and covariance matrix adaptation](http://nbviewer.ipython.org/github/lmarti/evolutionary-computation-course/blob/master/AEC.04%20-%20Evolutionary%20Strategies%20and%20Covariance%20Matrix%20Adaptation.ipynb), from the [Advanced Evolutionary Computation: Theory and Practice](http://nbviewer.ipython.org/github/lmarti/evolutionary-computation-course/tree/master/) course by [Luis Martí](http://lmarti.com).
### Statistics, Machine Learning and Data Science
* [AM207: Monte Carlo Methods, Stochastic Optimization](http://am207.org): a complete course by Verena Kaynig-Fittkau and Pavlos Protopapas from Harvard, with all lecture materials and homework sets as notebooks.
* [An introduction to Bayesian inference](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1.ipynb), this is just chapter 1 in an ongoing book titled [Probabilistic Programming and Bayesian Methods for Hackers Using Python and PyMC](http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers), by [Cameron Davidson-Pilon](http://camdp.com/).
* [Learn Data Science](http://learnds.com), an entire self-directed course by [Nitin Borwankar](https://github.com/nborwankar).
* [IPython Cookbook](http://ipython-books.github.io/cookbook/) by [Cyrille Rossant](http://cyrille.rossant.net/), a comprehensive guide to Python for Data Science. The code of the 100 recipes is available on [the GitHub repository](https://github.com/ipython-books/cookbook-code).
* [An introduction to machine learning with Python and scikit-learn](http://nbviewer.ipython.org/github/temporaer/tutorial_ml_gkbionics/blob/master/2%20-%20KMeans.ipynb) ([repo and overview](https://github.com/amueller/tutorial_ml_gkbionics)) by [Hannes Schulz](https://github.com/temporaer) and [Andreas Mueller](https://github.com/amueller).
* [Clustering and Regression](http://nbviewer.ipython.org/github/amplab/datascience-sp14/blob/master/hw2/HW2.ipynb), part of the UC Berkeley 2014 [Introduction to Data Science](http://amplab.github.io/datascience-sp14/) course taught by [Michael Franklin](http://www.cs.berkeley.edu/~franklin/).
* [Neural Networks](http://nbviewer.ipython.org/github/masinoa/machine_learning/blob/master/04_Neural_Networks.ipynb), part of a collection on [machine learning](https://github.com/masinoa/machine_learning) by [Aaron Masino](https://github.com/masinoa).
* [An introduction to Pandas](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/01%20-%20Lesson.ipynb), part of an [11-lesson tutorial on Pandas](https://bitbucket.org/hrojas/learn-pandas), by [Hernán Rojas](https://bitbucket.org/hrojas).
* The [Statsmodels Project](http://statsmodels.sourceforge.net) has two excellent collections of examples: [in their official documentation](http://statsmodels.sourceforge.net/devel/examples/index.html) and [extra ones in their wiki](https://github.com/statsmodels/statsmodels/wiki/Examples#user-contributed-examples). Too many there to directly duplicate here, but they provide great learning materials on statistical modeling with Python.
* [Machine Learning with the Shogun Toolbox](http://cloud.shogun-toolbox.org). This is a complete *service* that includes a ready-to-run IPython instance with a collection of notebooks illustrating the use of the [Shogun Toolbox](http://shogun-toolbox.org). Just log in and start running the examples.
* [Python for Data Analysis](https://github.com/ResearchComputing/Meetup-Fall-2013), an introductory collection from the [CU Boulder Research Computing Group](http://researchcomputing.github.io/).
* [The Kaggle bulldozers competition example](http://danielfrg.github.io/blog/2013/03/07/kaggle-bulldozers-basic-cleaning), one of a set on tutorials on exploratory data analysis with the [copper toolkit](https://github.com/danielfrg/copper#copper) by [Daniel Rodríguez](http://danielfrg.github.io)/
* [Understanding model reliability](http://nbviewer.ipython.org/github/mwaskom/Psych216/blob/master/week6_tutorial.ipynb), part of a complete [course on statistics and data analysis for psychologists](https://github.com/mwaskom/Psych216) by [Michael Waskom](https://github.com/mwaskom).
* [Graphical Representations of Linear Models](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/linear_models.ipynb), an illustration of the [Seaborn statistical visualization library](https://github.com/mwaskom/seaborn), that also includes [Visualizing distributions of data](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/plotting_distributions.ipynb) and [Representing variability in timeseries plots](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/timeseries_plots.ipynb). By [Michael Waskom](https://github.com/mwaskom).
* [Desperately Seeking Silver](http://nbviewer.ipython.org/github/cs109/content/blob/master/HW2.ipynb), one of the homework sets for Harvard's [CS 109 Data Science course](http://cs109.org).
* [IPython Notebooks for 'An Introduction to Statistical Learning with Applications in R'](https://github.com/JWarmenhoven/ISLR-python), Python code for a selection of tables, figures and LAB sections from the book by James, Witten, Hastie, Tibshirani (2013).
* [Python Notebooks for StatLearning Exercises](https://github.com/sujitpal/statlearning-notebooks), Python implementations of the R labs for the [StatLearning: Statistical Learning](https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/about) online course from Stanford University taught by Profs Trevor Hastie and Rob Tibshirani.
* [Applied Predictive Modeling with Python](http://nbviewer.ipython.org/github/leig/Applied-Predictive-Modeling-with-Python/tree/master/notebooks/), Python implementations of the examples (originally written in R) from a famous introductory book, [Applied Predictive Modeling](http://appliedpredictivemodeling.com), by Max Kuhn and Kjell Johnson.
* A collection of [four courses in foundations of data science, algorithms and databases](http://nbviewer.ipython.org/github/ledeprogram/courses/tree/master) from multiple faculty at [Columbia University's Lede Program](http://www.journalism.columbia.edu/page/1058-the-lede-program-an-introduction-to-data-practices/906).
* [SciPy and OpenCV as an interactive computing environment for computer vision](http://nbviewer.ipython.org/github/thsant/scipy4cv) by [Thiago Santos](http://ttsantos.net), a tutorial presented at [SIBGRAPI 2014](http://emap.fgv.br/sibgrapi-2014/tutorials.html).
* [Kalman and Bayesian Filters in Python](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb), by [Roger Labbe](https://github.com/rlabbe).
* [Machine learning in Python](http://nbviewer.ipython.org/github/jdwittenauer/ipython-notebooks/blob/master/ML-Exercise1.ipynb), a series based on Andrew Ng's Coursera class on machine learning. Part of a [larger collection of data science notebooks](https://github.com/jdwittenauer/ipython-notebooks) by [John Wittenauer](https://github.com/jdwittenauer).
* [An example machine learning notebook](http://nbviewer.jupyter.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/example-data-science-notebook/Example%20Machine%20Learning%20Notebook.ipynb), by [Randal. S. Olson](http://www.randalolson.com/), part of a [collection in Data Analysis and Machine Learning](http://nbviewer.jupyter.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects).
### Mathematics, Physics, Chemistry, Biology
* A [single-atom laser model](http://nbviewer.ipython.org/github/jrjohansson/qutip-lectures/blob/master/Lecture-2B-Single-Atom-Lasing.ipynb). This is one of a complete set of [lectures on quantum mechanics and quantum optics using QuTiP](http://nbviewer.ipython.org/github/jrjohansson/qutip-lectures) by [J.R. Johansson](https://github.com/jrjohansson).
* [2-d rigid-body transformations](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/Transformation2D.ipynb). This is part of [Scientific Computing in Biomechanics and Motor Control](https://github.com/demotu/BMC), a complete collection of notebooks by [Marcos Duarte](https://github.com/demotu).
* Astrophysical simulations and analysis with [yt](http://yt-project.org): a collection of example notebooks on using various codes that yt interfaces with: [Enzo](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/ac275ee8a462425c93b36e330e243705.ipynb), [Gadget](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9ab12c55422bb3afdc.ipynb), [RAMSES](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb), [PKDGrav](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb) and [Gasoline](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb). *Note:* the yt site currently throws an SSL warning, they seem to have an outdated or self-signed certificate.
* [Working with Reactions](http://nbviewer.ipython.org/gist/4316430), part of a set of tutorials on [cheminformatics and machine learning with the rdkit project](http://code.google.com/p/rdkit/wiki/UGM2012Tutorials), by Greg Landrum.
* [CFD Python: 12 steps to Navier-Stokes](http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes). A complete set of lectures on Computational Fluid Dynamics, from 1-d linear waves to full 2-d Navier-Stokes, by [Lorena Barba](http://lorenabarba.com).
* [AeroPython: Aerodynamics-Hydrodynamics with Python](https://github.com/barbagroup/AeroPython), a complete course taught at George Washington University by [Lorena Barba](http://lorenabarba.com).
* [Practical Numerical Methods with Python](https://github.com/numerical-mooc/numerical-mooc), a collection of learning modules (each consisting of several IPython Notebooks) for a course in numerical differential equations taught at George Washington University by [Lorena Barba](http://lorenabarba.com). Also offered as a "massive, open online course" (MOOC) on the [GW SEAS Open edX](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about) platform.
* [pyuvvis: tools for explorative spectroscopy](https://github.com/hugadams/pyuvvis), spectroscopy library built for integration ipython notebooks, matplotlib and pandas.
* [HyperPython: a practical introduction to the solution of hyperbolic conservation laws](http://nbviewer.ipython.org/github/ketch/HyperPython/tree/master/), a course by [David Ketcheson](http://davidketcheson.info).
* [An Introduction to Applied Bioinformatics](http://caporasolab.us/An-Introduction-To-Applied-Bioinformatics): Interactive lessons in bioinformatics, by [Greg Caporaso](http://caporasolab.us).
* Colour science computations with [colour](https://github.com/colour-science/colour), a Python package implementing a comprehensive number of colour theory transformations and algorithms supported by a [dedicated collection of IPython Notebooks](http://nbviewer.ipython.org/github/colour-science/colour-ipython/blob/master/notebooks/colour.ipynb). More colour science related [IPython Notebooks](http://nbviewer.ipython.org/github/colour-science/colour-website/tree/master/ipython/) are available on [colour-science.org](http://colour-science.org/).
* The [notebooks](https://github.com/tiagoantao/bioinf-python/blob/master/notebooks/Welcome.ipynb) from the Book [Bioinformatics with Python Cookbook](http://www.amazon.com/Bioinformatics-Python-Cookbook-Tiago-Antao/dp/1782175113), covering several fields like Next-Generation Sequencing, Population Genetics, Phylogenetics, Genomics, Proteomics and Geo-referenced information.
### Earth Science and Geo-Spatial data
* [EarthPy](http://earthpy.org), a collection of IPython notebooks with a focus on Earth Sciences, from [whale tracks](http://earthpy.org/analyzing-whale-tracks.html) to the [flow of the Amazon](http://earthpy.org/earthpy-basemap-amazon.html).
* [Python for Geosciences](https://github.com/koldunovn/python_for_geosciences), a tutorial series aimed at the Earth Sciences community, by [Nikolay Koldunov](http://earthpy.org/author/nikolay-koldunov.html).
* [Find graffiti close to NY subway entrances](http://nbviewer.ipython.org/github/invisibleroads/crosscompute-tutorials/blob/master/computational-analysis/300%20Count%20graffiti%20sightings%20within%20100%20feet%20of%20a%20subway%20entrance.ipynb), one of a rich [collection of notebooks on large-scale data analysis](https://github.com/invisibleroads/crosscompute-tutorials), by [Roy Hyunjin Han](https://github.com/invisibleroads).
* [Logistic models of well switching in Bangladesh](http://nbviewer.ipython.org/github/carljv/Will_it_Python/blob/master/ARM/ch5/arsenic_wells_switching.ipynb), part of the ["Will it Python"](http://slendrmeans.wordpress.com/will-it-python) blog series ([repo](https://github.com/carljv/Will_it_Python)) on Machine Learning and data analysis in Python. By Carl Vogel.
* [Estimated likelihood of observing a large earthquake on a continental lowangle normal fault and implications for lowangle normal fault activity](http://nbviewer.ipython.org/github/cossatot/lanf_earthquake_likelihood/blob/master/notebooks/lanf_manuscript_notebook.ipynb), an executable version of a paper by Richard Styron and Eric Hetland published in *Geophysical Research Letters*, on earthquake probabilities.
* [python4oceanographers](https://ocefpaf.github.io/python4oceanographers/), a blog demonstrating analyses in physical oceanography from [resource-demanding numerical computations with functions in compiled languages](https://ocefpaf.github.io/python4oceanographers/blog/2015/10/05/isosurfaces/) to specialized [tidal analysis](https://ocefpaf.github.io/python4oceanographers/blog/2015/02/16/utide/) to [visualization of various geo data using fancy things like interactive maps](https://ocefpaf.github.io/python4oceanographers/blog/2015/07/13/interactive_geo/).
### Linguistics and Text Mining
* [Workshop on text analysis](http://nbviewer.ipython.org/github/nealcaren/workshop_2014/tree/master/notebooks/) by [Neal Caren](http://nealcaren.web.unc.edu).
* [Detecting Algorithmically Generated Domains](http://nbviewer.ipython.org/github/ClickSecurity/data_hacking/blob/master/dga_detection/DGA_Domain_Detection.ipynb), part of the [Data Hacking](http://clicksecurity.github.io/data_hacking) collection on security-oriented data analysis with IPython & friends.
* [Mining the Social Web (2nd Edition)](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition). A complete collection of notebooks accompanying [Matthew Russel's book](http://shop.oreilly.com/product/0636920030195.do) by O'Reilly.
### Signal Processing
* [Sound Analysis with the Fourier Transform](https://github.com/calebmadrigal/FourierTalkOSCON). A set of IPython Notebooks by [Caleb Madrigal](http://calebmadrigal.com) to explain what the Fourier Transform is and how to use it for basic audio processing applications.
* [An introduction to Compressed Sensing](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Compressive_Sampling.ipynb), part of [Python for Signal Processing](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing): an entire book (and [blog](http://python-for-signal-processing.blogspot.com)) on the subject by Jose Unpingco.
* [Kalman and Bayesian Filters in Python](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb). A textbook and accompanying filtering library on the topic of Kalman filtering and other related Bayesian filtering techniques.
* [Classify human movements using Dynamic Time Warping & K Nearest Neighbors:](http://nbviewer.ipython.org/github/markdregan/K-Nearest-Neighbors-with-Dynamic-Time-Warping/blob/master/K_Nearest_Neighbor_Dynamic_Time_Warping.ipynb) Signals from a smart phone gyroscope and accelerometer are used to classify if the person is running, walking, sitting standing etc. This IPython notebook contains a python implementation of DTW and KNN algorithms along with explanations and a practical application.
* [Digital Signal Processing](https://github.com/spatialaudio/digital-signal-processing-lecture) A collection of notebooks that accompanies a masters course on the topic.
### Engineering Education
* [Introduction to Chemical Engineering Analysis](http://jckantor.github.io/CBE20255/) by [Jeff Kantor](http://jckantor.github.io/). A collection of IPython notebooks illustrating topics in introductory chemical engineering analysis, including stoichiometry, generation-consumption analysis, mass and energy balances.
## Scientific computing and data analysis with the SciPy Stack
### General topics in scientific computing
* [Algorithms in IPython notebooks](https://github.com/rasbt/algorithms_in_ipython_notebooks), by [Sebastian Raschka](https://github.com/rasbt)
* [Comparing the performance of Python compilers - Cython vs. Numba vs. Parakeet](_nbs/day4_2_cython_numba_parakeet.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt)
* [A Crash Course in Python for Scientists](http://nbviewer.ipython.org/gist/5920182), by Sandia's [Rick Muller](http://www.cs.sandia.gov/~rmuller).
* [A gentle introduction to scientific programming in Python, biased towards biologists](http://nbviewer.ipython.org/url/atwallab.cshl.edu/teaching/QBbootcamp3.ipynb), by [Mickey Atwal, Cold Spring Harbor Laboratory](http://atwallab.cshl.edu).
* [Python for Data Science](http://nbviewer.ipython.org/github/gumption/Python_for_Data_Science/blob/master/Python_for_Data_Science_all.ipynb), a self-contained mini-course with exercises, by [
Joe McCarthy](http://interrelativity.com/joe).
* [First few lectures of the UW/Coursera course on Data Analysis](http://nbviewer.ipython.org/github/fonnesbeck/ComputationalMethodsCourse/blob/master/Lecture%201.ipynb). ([Repo](https://github.com/fonnesbeck/ComputationalMethodsCourse)) by Chris Fonnesbeck.
* [CythonGSL: a Cython interface for the GNU Scientific Library (GSL)](http://nbviewer.ipython.org/github/twiecki/CythonGSL/blob/master/examples/cython_gsl_ipythonnb.ipynb) ([Project repo](https://github.com/twiecki/CythonGSL), by Thomas Wiecki.
* [Using Numba to speed up numerical codes](http://nbviewer.ipython.org/gist/3914904). And another Numba example: [self-organizing maps](http://nbviewer.ipython.org/gist/3407544).
* [Numpy performance tricks](http://nbviewer.ipython.org/gist/4645217), and [blog post](http://cyrille.rossant.net/numpy-performance-tricks/), by [Cyrille Rossant](http://cyrille.rossant.net/).
* [IPython Parallel Push/Execute/Pull Demo](http://nbviewer.ipython.org/gist/3866987) by Justin Riley.
* [Understanding the design of the R "formula" objects](http://nbviewer.ipython.org/url/perrin.dynevor.org/exploring_r_formula.ipynb) by Matthew Brett.
* [Comparing different approaches to evolutionary simulations](http://nbviewer.ipython.org/url/www.slideviper.oquanta.info/test/simulation%20frameworks.ipynb). Also available [here](http://slideviewer.herokuapp.com/urls/github.com/yoavram/ipython-notebooks/raw/master/simulation%20frameworks.ipynb) to better visualization. The notebook was converted to a HTML presentation using an old nbconvert with the first developing implementation of `reveal` converter. By Yoav Ram.
* [The Traveling Salesperson Problem](http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipynb) by [Peter Norvig](http://norvig.com).
* [A git tutorial targeted at scientists](http://nbviewer.ipython.org/github/fperez/reprosw/blob/master/Version%20Control.ipynb) by Fernando Perez.
* [Running MATLAB in an IPython Notebook](http://nbviewer.ipython.org/github/plotly/IPython-plotly/blob/master/See%20more/MATLAB_Wrapper_Examples.ipynb), using [pymatbridge](https://pypi.python.org/pypi/pymatbridge).
* [Interactive Curve-Fitting](http://nbviewer.ipython.org/gist/danielballan/1c2db3d4f2f7780cf52f) The `lmfit` package provides a widget-based interface to the curve-fitting algorithms in SciPy.
* [A visual guide to the Python Spark API for distributed computing](http://nbviewer.ipython.org/github/jkthompson/pyspark-pictures/blob/master/pyspark-pictures.ipynb) by Jeff Thompson
### Social data
* [Survival Analysis](http://nbviewer.ipython.org/github/CamDavidsonPilon/lifelines/blob/master/docs/Survival%20Analysis%20intro.ipynb), an illustration of the [lifelines library](https://github.com/CamDavidsonPilon/lifelines), by [Cam Davidson Pilon](https://github.com/CamDavidsonPilon).
* A reconstruction of [Nate Silver's 538 model for the 2012 US Presidential Election](http://nbviewer.ipython.org/github/jseabold/538model/blob/master/silver_model.ipynb), by Skipper Seabold (complete [repo](https://github.com/jseabold/538model)).
* [Data about the Sandy Hook massacre in Newtown, Conneticut](http://nbviewer.ipython.org/url/www.brianckeegan.com/data/Shootings/Shootings.ipynb), which accompanies a more detailed [blog post on the subject](http://www.brianckeegan.com/2012/12/sandy-hook-school-massacre/). Here are the [notebook and accompanying data](http://www.brianckeegan.com/data/Shootings/Shootings.ipynb). By Brian Keegan.
* More on [gun violence analysis with Wikipedia data](http://nbviewer.ipython.org/gist/4358066).
* [An analysis of the Gaza-Israel 2012 crisis](http://nbviewer.ipython.org/gist/4121857).
* [Ranking NFL Teams](http://nbviewer.ipython.org/github/seanjtaylor/NFLRanking/blob/master/NFL%20Rankings.ipynb). The [full repo](https://github.com/seanjtaylor/NFLRanking) also includes an explanatory slideshow. By Sean Taylor.
* [Automated processing of news media and generation of associated imagery](http://nbviewer.ipython.org/url/mhermans.net/files/tmp/demo_rdf_HLN.ipynb).
* [An analysis of national school standardized test data in Colombia using Pandas](http://nbviewer.ipython.org/url/finiterank.com/saber/saber.ipynb) (in Spanish). By [Javier Moreno](http://finiterank.com).
* [Getting started with GDELT](http://nbviewer.ipython.org/github/dmasad/GDELT_Intro/blob/master/Getting_Started_with_GDELT.ipynb), by [David Masad](https://github.com/dmasad). [GDELT](http://eventdata.psu.edu/data.dir/GDELT.html) is a dataset containing more than 200-million geolocated events with global coverage for 1979 to the present. Another GDELT example from David, that nicely [integrates mapping visualizations](http://nbviewer.ipython.org/github/dmasad/GDELT_Intro/blob/master/GDELT_Mapping.ipynb).
* [Titanic passengers, coal mining disasters, and vessel speed changes](http://nbviewer.ipython.org/gist/fonnesbeck/8495259), by [Christopher Fonnesbeck](http://biostat.mc.vanderbilt.edu/wiki/Main/ChrisFonnesbeck)
* A geographic analysis of [Indonesian conflicts in 2012](http://nbviewer.ipython.org/github/herrfz/gdelt/blob/master/indn_connection.ipynb) with GDELT, by [herrfz](https://github.com/herrfz).
* [Bioinformatic Approaches to the Computation of Poetic Meter](http://nbviewer.ipython.org/github/asp49/meter/blob/graph/Shared%20Horizons%20Presentation.ipynb), by [A. Sean Pue](https://github.com/asp49), [C. Titus Brown](https://github.com/ctb) and [Tracy Teal](https://github.com/tracyteal).
* Analyzing the [Vélib dataset from Paris](http://nbviewer.ipython.org/gist/5520933), by [Cyrille Rossant](https://github.com/rossant) (Vélib is Paris' [bicycle-sharing program](http://en.wikipedia.org/wiki/V%C3%A9lib')).
* [Using Python to see how the Times writes about men and women](http://nbviewer.ipython.org/gist/5105037), by [Neal Caren](https://twitter.com/HaphazardSoc).
* [Exploring graph properties of the Twitter stream with twython and NetworkX](http://nbviewer.ipython.org/gist/5681541/TwitterGraphs.ipynb), by [F. Perez](http://fperez.org) (complete [gist repo with utilities here](https://gist.github.com/fperez/5681541).)
* [Kaggle Competition: Titanic Machine Learning from Disaster](http://nbviewer.ipython.org/github/agconti/kaggle-titanic/blob/master/Titanic.ipynb). By [Andrew Conti](https://github.com/agconti).
* [How clean are San Francisco's restaurants?](http://nbviewer.ipython.org/github/Jay-Oh-eN/happy-healthy-hungry/blob/master/h3.ipynb), a data science tutorial that accompanies a [blog post](http://blog.zipfianacademy.com/post/57158627293/how-to-data-science-mapping-sf-restaurant-inspection) from [Zipfian Academy](http://zipfianacademy.com/).
* [NYT gender wage gap and US crime by state](http://nbviewer.ipython.org/github/plotly/IPython-plotly/blob/master/Bubble%20Chart%20Explorer.ipynb).
* [Predicting usage of the subway system in NYC](http://nbviewer.ipython.org/url/www.asimihsan.com/articles/Intro%20to%20Data%20Science%20-%20Final%20Project.ipynb), a [final project](http://blog.udacity.com/2014/05/intro-to-data-science-tools-to-ask.html) for the Udacity Intro to Data Science Course, by [Asim Ihsan](http://www.asimihsan.com).
* [An exploratory statistical analysis of the 2014 World Cup Final](http://nbviewer.ipython.org/github/rjtavares/football-crunching/blob/master/notebooks/an%20exploratory%20data%20analysis%20of%20the%20world%20cup%20final.ipynb), by [Ricardo Tavares](https://github.com/rjtavares). Part of a [notebook collection on football (aka soccer) analysis](http://nbviewer.ipython.org/github/rjtavares/football-crunching/tree/master/).
* [San Francisco's Drug Geography](http://nbviewer.ipython.org/github/lmart999/GIS/blob/master/SF_GIS_Crime.ipynb), a GIS analysis of public crime data in SF, by [Lance Martin](https://github.com/lmart999).
### Psychology and Neuroscience
* [Cue Combination with Neural Populations](http://nbviewer.ipython.org/github/wtadler/cue-combination-with-neurons/blob/master/neural_cue_combination.ipynb) by [Will Adler](http://www.wtadler.com). Intuition and simulation for the theory (Ma *et al.*, 2006) that through probabilistic population codes, neurons can perform optimal cue combination with simple linear operations. Demonstrates that variance in cortical activity, rather than impairing sensory systems, is an adaptive mechanism to encode uncertainty in sensory measurements.
* [Modeling psychophysical data with non-linear functions](http://nbviewer.ipython.org/github/arokem/teach_optimization/blob/master/optimization.ipynb) by Ariel Rokem.
* [Visualizing mathematical models of brain cell connections](http://nbviewer.ipython.org/github/jonasnick/ReceptiveFields/blob/master/receptiveFields.ipynb). The effect of convolution of different receptive field functions and natural images is examined.
* [Python for Vision Research](http://nbviewer.ipython.org/github/gestaltrevision/python_for_visres/blob/master/index.ipynb). A three-day crash course for vision researchers in programming with Python, building experiments with [PsychoPy](http://psychopy.org/) and [psychopy_ext](http://psychopy_ext.klab.lt/), learning the fMRI multi-voxel pattern analysis with [PyMVPA](http://www.pymvpa.org/), and understading image processing in Python.
* [Loading and visualizing fMRI data](http://nbviewer.ipython.org/github/GaelVaroquaux/nilearn_course/blob/master/rendered_notebooks/1_Introduction.ipynb), part of the [Functional connectivity with NiLearn course](https://github.com/GaelVaroquaux/nilearn_course) by [Gaël Varoquaux](http://gael-varoquaux.info).
### Machine Learning, Statistics and Probability
* [An introduction to parallel machine learning with sklearn, joblib and IPython.parallel](http://nbviewer.ipython.org/gist/5115540/Model%20Selection%20for%20the%20Nystroem%20Method.ipynb), a notebook that accompanies this [slide deck](https://speakerdeck.com/ogrisel/parallel-and-large-scale-machine-learning-with-scikit-learn) by [Olivier Grisel](https://github.com/ogrisel).
* [A tutorial introduction to machine learning with sklearn](http://amueller.github.com/sklearn_tutorial), an IPython-based slide deck by [Andreas Mueller](https://github.com/amueller).
* [Introduction to Machine Learning in Python with scikit-learn](http://ipython-books.github.io/featured-04/) by [Cyrille Rossant](http://cyrille.rossant.net/), a free recipe from the [IPython Cookbook](http://ipython-books.github.io/cookbook/), a comprehensive guide to Python for Data Science.
* [An introduction to Predictive Modeling in Python](http://nbviewer.ipython.org/github/ogrisel/parallel_ml_tutorial/blob/master/solutions/01%20-%20An%20Introduction%20to%20Predictive%20Modeling%20in%20Python.ipynb), by [Olivier Grisel](https://github.com/ogrisel).
* [Face Recognition on a subset of the Labeled Faces in the Wild dataset](http://nbviewer.ipython.org/github/ogrisel/notebooks/blob/master/Labeled%20Faces%20in%20the%20Wild%20recognition.ipynb), by [Olivier Grisel](http://github.com/ogrisel).
* [An Introduction to Bayesian Methods for Multilevel Modeling](http://nbviewer.ipython.org/github/fonnesbeck/multilevel_modeling/blob/master/multilevel_modeling.ipynb), by [Chris Fonnesbeck](http://github.com/fonnesbeck).
* [Introduction to Bayesian Networks](http://nbviewer.ipython.org/github/kuitang/hackny-bayesnet/blob/master/hackNY%20Bayesian%20Network%20Demo.ipynb) by [Kui Tang](https://github.com/kuitang)
* [Bayesian data analysis with PyMC3](http://nbviewer.ipython.org/github/twiecki/pymc3_talk/blob/master/bayesian_pymc3.ipynb) by [Thomas Wiecki](https://github.com/twiecki).
* [A collection of examples for solving pattern classification problems](https://github.com/rasbt/pattern_classification), by [Sebastian Raschka](https://github.com/rasbt).
* [Introduction to Linear Regression using Python](http://nbviewer.ipython.org/github/justmarkham/DAT4/blob/master/notebooks/08_linear_regression.ipynb) by [Kevin Markham](https://github.com/justmarkham)
* [Machine learning in Python](http://nbviewer.ipython.org/github/jdwittenauer/ipython-notebooks/blob/master/ML-Exercise1.ipynb), a series based on Andrew Ng's Coursera class on machine learning. Part of a [larger collection of data science notebooks](https://github.com/jdwittenauer/ipython-notebooks) by [John Wittenauer](https://github.com/jdwittenauer).
* [Probability, Paradox, and the Reasonable Person Principle](http://nbviewer.ipython.org/url/norvig.com/ipython/Probability.ipynb), by Peter Norvig.
### Physics, Chemistry and Biology
* [Multibody dynamics and control with Python](http://www.moorepants.info/blog/npendulum.html) and the [notebook file](http://nbviewer.ipython.org/github/gilbertgede/idetc-2013-paper/blob/master/n-pendulum-control.ipynb) by Jason K. Moore.
* [Manipulation and display of chemical structures](http://nbviewer.ipython.org/gist/4316433), by [Greg Landrum](https://github.com/greglandrum), using rdkit.
* [The sound of Hydrogen](http://nbviewer.ipython.org/github/Carreau/posts/blob/master/07-the-sound-of-hydrogen.ipynb), visualizing and listening to the quantum-mechanical spectrum of Hydrogen. By [Matthias Bussonnier](http://github.com/Carreau).
* Particle physics at the Large Hadron Collider (LHC): using [ROOT in an LHCb masterclass](http://root.cern.ch/drupal/content/ipython-notebooks-and-root-0): [Notebook 1](http://nbviewer.ipython.org/github/mazurov/webfest2013/blob/master/notebooks/MasterClassD0-ex1.ipynb) and [Notebook 2](http://nbviewer.ipython.org/github/mazurov/webfest2013/blob/master/notebooks/MasterClassD0-ex2%2Cex3.ipynb) notebooks by Alexander Mazurov and Andrey Ustyuzhanin at CERN.
* [A Reaction-Diffusion Equation Solver in Python with Numpy](http://nbviewer.ipython.org/github/waltherg/notebooks/blob/master/2013-12-03-Crank_Nicolson.ipynb), a demonstration of how IPython notebooks can be used to discuss both the theory and implementation of numerical algorithms on one page, by [Georg Walther](https://github.com/waltherg).
* [Comparing different approaches to evolutionary simulations](http://nbviewer.ipython.org/url/www.slideviper.oquanta.info/test/simulation%20frameworks.ipynb). Also available [here](http://slideviewer.herokuapp.com/urls/github.com/yoavram/ipython-notebooks/raw/master/simulation%20frameworks.ipynb) to better visualization. The notebook was converted to a HTML presentation using an old nbconvert with the first developing implementation of `reveal` converter. By Yoav Ram.
### Economics and Finance
* [Replication of the highly-contentious analysis of economic growth by Reinhart and Rogoff](http://nbviewer.ipython.org/github/vincentarelbundock/Reinhart-Rogoff/blob/master/reinhart-rogoff.ipynb), by [Vincent Arel-Bundock](http://umich.edu/~varel), [full repo here](https://github.com/vincentarelbundock/Reinhart-Rogoff). This is based on the [widely-publicized critique of the original analysis done by Herndon, Ash, and Pollin](http://www.peri.umass.edu/236/hash/31e2ff374b6377b2ddec04deaa6388b1/publication/566).
* [fecon235 for Financial Economics](https://github.com/rsvp/fecon235) series of notebooks which examines time-series data for economics and finance. Easy API to freely access data from the Federal Reserve, SEC, CFTC, stock and futures exchanges. Thus research from older notebooks can be replicated, and updated using the most current data. For example, this notebook forecasts likely Fed policy for setting the [Fed Funds rate](https://git.io/fedfunds), but market sentiment across major asset classes is observable from the [CFTC Commitment of Traders Report](https://git.io/cotr). Major economics indicators are renormalized: for example, various measures of [inflation](https://git.io/infl), optionally with the forward-looking break-even rates derived from U.S. Treasury bonds. Other notebooks examine international markets: especially, gold and foreign exchange.
### Earth science and geo-spatial data
* [Exploring seafloor habitats: geographic analysis using IPython Notebook with GRASS & R](http://nbviewer.ipython.org/gist/7598354). This embeds a slideshow and a Web Spinning Globe (Cesium) in the notebook. By Massimo Di Stefano.
* [Geo-Spatial Data with IPython](http://nbviewer.ipython.org/github/mqlaql/geospatial-data/blob/master/Geospatial-Data-with-Python.ipynb). Tutorial by [Kelsey Jordahl](http://kjordahl.net/) from SciPy2013.
### Data visualization and plotting
* A [Notebook](http://nbviewer.ipython.org/github/plotly/python-user-guide/blob/master/s3_bubble-charts/s3_bubble-charts.ipynb) with an interactive Hans Rosling Gapminder bubble chart from [Plotly](https://plot.ly/api/Python).
* [Data and visualization integration via web based resources](http://tw.rpi.edu/media/2013/09/25/a48/The_Perfect_Storm_1991.html). Using NetCDF, Matplotlib, IPython Parallel and ffmpeg to generate video animation from time series of gridded data. By Massimo Di Stefano.
* [21 Interactive, D3 Plots from matplotlib, ggplot for Python,
prettyplotlib, Stack Overflow, and seaborn](http://nbviewer.ipython.org/gist/msund/7ac1203ded66fe8134cc).
* [Visualizing complex-valued functions with Matplotlib and Mayavi](http://nbviewer.ipython.org/github/empet/Math/blob/master/DomainColoring.ipynb), by [Emilia Petrisor](https://github.com/empet).
* [A D3 Viewer for Matplotlib Visualizations](http://jakevdp.github.io/blog/2013/12/19/a-d3-viewer-for-matplotlib/), different from above by not depending on Plot.ly account.
* [Bokeh is an interactive web visualization library for Python](http://nbviewer.ipython.org/github/damianavila/bokeh_overview/blob/master/Bokeh%20Overview.ipynb?create=1) (and other languages). It provides d3-like novel graphics, over large datasets, all without requiring any knowledge of Javascript. It also has a Matplotlib compatibility layer.
* [Winner of the 2014 E. Tufte Slope Graphs contest](http://nbviewer.ipython.org/gist/pascal-schetelat/8382651), by [Pascal Schetelat](https://github.com/pascal-schetelat). The original [contest info on Tufte's site](http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003nk).
* [matta, d3.js-based visualizations in the IPython Notebook](http://nbviewer.ipython.org/github/carnby/matta/blob/master/examples/Basic%20Examples.ipynb), by [Eduardo Graells-Garrido](http://carnby.github.io/).
### Mathematics
* [Linear algebra with Cython](http://nbviewer.ipython.org/github/carljv/cython_testing/blob/master/cython_linalg.ipynb). A tutorial that styles the notebook differently to show that you can produce high-quality typography online with the Notebook. By Carl Vogel.
* [Exploring how smooth-looking functions can have very surprising derivatives even at low orders](http://nbviewer.ipython.org/url/finiterank.com/cuadernos/suavesylocas.ipynb), combining SymPy and matplotlib. By [Javier Moreno](http://finiterank.com).
* [A Collection of Applied Mathematics and Machine Learning Tutorials](http://sayilarvekuramlar.blogspot.co.uk/2015/12/matematik-ders-notlari.html) (in Turkish). By Burak Bayramli.
* [Function minimization with iminuit](http://nbviewer.ipython.org/github/iminuit/iminuit/blob/master/tutorial/tutorial.ipynb), an introductory companion to their [hard core tutorial](http://nbviewer.ipython.org/github/iminuit/iminuit/blob/master/tutorial/hard-core-tutorial.ipynb). By the [iminuit project](http://iminuit.github.io/iminuit).
* [The Discrete Cosine Transform](http://nbviewer.ipython.org/url/cs.marlboro.edu/courses/spring2014/information/code/dct/dct.ipynb), a brief explanation and illustration of the math behind the DCT and its role in the JPEG image format, by [Jim Mahoney](http://cs.marlboro.edu).
* [Chebfun in Python](http://nbviewer.ipython.org/gist/6724986), a demo of [PyChebfun](https://github.com/cswiercz/pychebfun), by [Olivier Verdier](http://www.olivierverdier.com). PyChebfun is a pure-python implementation of the celebrated [Chebfun package by Battles and Trefethen](http://people.maths.ox.ac.uk/trefethen/publication/PDF/2004_107.pdf).
* [The Matrix Exponential]
(http://nbviewer.ipython.org/github/sdrelton/matrix_function_notebooks/blob/master/TheMatrixExponential.ipynb), an introduction to the matrix exponential, its applications, and a list of available software in Python and MATLAB. By [Sam Relton](http://www.maths.manchester.ac.uk/~srelton/).
* [Fractals, complex numbers, and your imagination](http://nbviewer.ipython.org/github/cfangmeier/ipython_notebooks/blob/master/Imagination.ipynb), by [Caleb Fangmeier](https://github.com/cfangmeier).
* [A SymPy tutorial](http://nbviewer.ipython.org/url/www.inp.nsk.su/~grozin/python/sympy.ipynb), by [Andrey Grozin](http://www.inp.nsk.su/~grozin/).
### Signal and Sound Processing
* [Simulation of Delta Sigma modulators in Python](http://nbviewer.ipython.org/github/ggventurini/python-deltasigma/blob/master/examples/dsdemo1.ipynb) with [deltasigma](https://github.com/ggventurini/python-deltasigma), Python port of of Richard Schreier's *excellent* [MATLAB Delta Sigma Toolbox](http://www.mathworks.com/matlabcentral/fileexchange/19-delta-sigma-toolbox), by [Giuseppe Venturini](https://github.com/ggventurini). Several demonstrative notebooks on the package [README](https://github.com/ggventurini/python-deltasigma/blob/master/README.md).
* [PyOracle: Automatic analysis of musical structure](http://nbviewer.ipython.org/urls/bitbucket.org/pucktronix/pyoracle/raw/d046b2bcf473503fa356094cfe4cff774d1aaefc/270D.ipynb), by [Greg Surges](http://gregsurges.com).
* [A Gallery of SciPy's Window Functions for quick visual inspection and comparison] (http://nbviewer.ipython.org/urls/gist.githubusercontent.com/jaidevd/b7d865f7f4b237ab5181/raw/30bc8f998bf8f924b56b32ce10acce125656ed7c/scipy_window_gallery.ipynb)
by [Jaidev Deshpande](http://twitter.com/jaidevd)
### Natural Language Processing
* [Python Programming for the Humanities](http://fbkarsdorp.github.io/python-course/) by Folgert Karsdorp & Maarten van Gompel.
### Pandas for data analysis
Note that in the 'collections' section above there are also pandas-related links, such as the one for an [11-lesson tutorial](https://bitbucket.org/hrojas/learn-pandas).
* [A 10-minute whirlwind tour of pandas](http://nbviewer.ipython.org/gist/wesm/4757075/PandasTour.ipynb), this is the notebook accompanying a [video presentation](http://vimeo.com/59324550) by Wes McKinney, author of Pandas and the [Python for Data Analysis](http://www.amazon.com/Python-Data-Analysis-Wes-McKinney/dp/1449319793) book.
* [Time-series analysis with Pandas](http://nbviewer.ipython.org/github/changhiskhan/talks/blob/master/pydata2012/pandas_timeseries.ipynb).
* [Financial data analysis with Pandas](http://nbviewer.ipython.org/gist/3962843).
* [Clustering of smartphone sensor data for human activity detection using pandas and scipy](http://nbviewer.ipython.org/github/herrfz/dataanalysis/blob/master/week4/clustering_example.ipynb), part of Coursera data analysis course, done in Python ([repo](https://github.com/herrfz/dataanalysis)).
* [Log analysis with Pandas](http://nbviewer.ipython.org/url/taaviburns.ca/presentations/log_analysis_with_pandas/nb/5-Scatterplots.ipynb), part of a [group presented at PyConCa 2012](http://taaviburns.ca/presentations/log_analysis_with_pandas/) by Taavi Burns.
* [Analyzing and visualizing sun spot data with Pandas](http://nbviewer.ipython.org/gist/4569783), by [Josh Hemann](https://github.com/jhemann). An enlightening discussion of how naive plotting choices subtly influence our interpretation of data.
* [Advanced analysis of Apache logs](http://nbviewer.ipython.org/github/koldunovn/nk_public_notebooks/blob/master/Apache_log.ipynb), by [Nikolay Koldunov](https://github.com/koldunovn).
* [Statistical Data Analysis in Python] (https://github.com/fonnesbeck/statistical-analysis-python-tutorial), by [Christopher Fonnesbeck](https://github.com/fonnesbeck/), SciPy 2013. Companion videos [1](https://www.youtube.com/watch?v=DXPwSiRTxYY), [2](https://www.youtube.com/watch?v=TGEBpzJUxdI), [3](https://www.youtube.com/watch?v=YZDtBEEZuAk), [4](https://www.youtube.com/watch?v=5_rcdhBXD-0)
## General Python Programming
* [Learning to code with Python](http://nbviewer.ipython.org/urls/bitbucket.org/amjoconn/watpy-learning-to-code-with-python/raw/3441274a54c7ff6ff3e37285aafcbbd8cb4774f0/notebook/Learn%20to%20Code%20with%20Python.ipynb), part of an [introduction to Python](https://bitbucket.org/amjoconn/watpy-learning-to-code-with-python/src) from the [Waterloo Python users group](http://watpy.ca/blog/post/learn-code-python-review-feb-2013).
* [Python Descriptors Demystified](http://nbviewer.ipython.org/gist/ChrisBeaumont/5758381/descriptor_writeup.ipynb), an in-depth discussion of the descriptor protocol in Python, by [Chris Beaumont](http://chrisbeaumont.org).
* [A collection of not so obvious Python stuff you should know!](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/not_so_obvious_python_stuff.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt).
* [Key differences between Python 2.7.x and Python 3.x](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/key_differences_between_python_2_and_3.ipynb), by [Sebastian Raschka](https://github.com/rasbt).
* [A beginner's guide to Python's namespaces, scope resolution, and the LEGB rule](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/scope_resolution_legb_rule.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt).
* [Sorting CSV files using the Python csv module](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/sorting_csvs.ipynb), by [Sebastian Raschka](https://github.com/rasbt).
* Python 3 OOP series by [Leonardo Giordani](https://github.com/lgiordani): [Part 1: Objects and types](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_1__Objects_and_types.ipynb), [Part 2: Classes and members](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_2__Classes_and_members.ipynb), [Part 3: Delegation - composition and inheritance](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_3__Delegation__composition_and_inheritance.ipynb), [Part 4: Polymorphism](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_4__Polymorphism.ipynb), [Part 5: Metaclasses](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_5__Metaclasses.ipynb), [Part 6: Abstract Base Classes](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_6__Abstract_Base_Classes.ipynb)
## Notebooks in languages other than Python
These are notebooks that use [one of the IPython kernels for other languages](IPython kernels for other languages):
### Julia
The IPython protocols to communicate between kernels and clients are language agnostic, and other programming language communities have started to build support for this protocol in their language. The Julia team has created [IJulia](https://github.com/JuliaLang/IJulia.jl), and these are some Julia notebooks:
* [Fractals 3 ways](http://nbviewer.ipython.org/url/beowulf.csail.mit.edu/18.337/fractals.ipynb), by [Jeff Bezanson](https://github.com/JeffBezanson).
* [The Design Impact of Multiple Dispatch](http://nbviewer.ipython.org/gist/StefanKarpinski/b8fe9dbb36c1427b9f22), a detailed explanation of Julia's multiple dispatch design, by [Stefan Karpinski](https://github.com/StefanKarpinski).
* A [tutorial](http://nbviewer.ipython.org/gist/7551139) on making interactive graphs with [Plotly](https://plot.ly/) and Julia.
* [Numerical tours in Julia](http://www.numerical-tours.com/julia/)
* [Functional Geometry](http://nbviewer.ipython.org/github/shashi/ijulia-notebooks/blob/master/funcgeo/Functional%20Geometry.ipynb) by [Shashi Gowda](https://github.com/shashi)
* [JuliaOpt notebooks](http://nbviewer.ipython.org/github/JuliaOpt/juliaopt-notebooks/tree/master/notebooks/), a collection of optimization-related notebooks.
* Coursework using IJulia notebooks:
- [Métodos Numéricos Avanzados (2015-2)](https://github.com/dpsanders/MetodosNumericosAvanzados), Luis Benet and David P. Sanders
- [Métodos Monte Carlo](https://github.com/dpsanders/metodos-monte-carlo), David Sanders
- [Linear Partial Differential Equations: Analysis and Numerics](http://math.mit.edu/~stevenj/18.303/), Steven G. Johnson
- [Julia tutorial for Computational Molecular Biology](http://cs.brown.edu/courses/csci1810/Julia_Tutorial.html), Younhun Kim and Matthew Reyna
* Other collections of IJulia notebooks:
- [Jiahao Chen](http://jiahao.github.io/code/)
- [Christoph Ortner](https://homepages.warwick.ac.uk/staff/C.Ortner/index.php?page=julia)
- [Crossing Language Barriers with Julia, Scipy, and IPython](https://github.com/stevengj/Julia-EuroSciPy14), presented at EuroSciPy '14 by Steven G. Johnson.
### Haskell
There exists a Haskell kernel for IPython in the [IHaskell project](https://github.com/gibiansky/ihaskell).
* [IHaskell Demo Notebook](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/IHaskell.ipynb)
* [Homophone reduction](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/Homophones.ipynb), a solution to a cute problem involving treating English letters as generators of a large group.
* [Gradient descent typeclass](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/Gradient-Descent.ipynb), a look at how arbitrary gradient descent algorithms can be represented with a typeclass.
### OCaml
[iocaml](https://github.com/andrewray/iocaml) is an OCaml kernel for IPython
* [H.261 Video Decoding in OCaml](https://andrewray.github.io/iocamljs/oh261.html)
* [OCaml implementation of the 2048 game](http://gazagnaire.org/fuconf14/)
### Ruby
Similar to the Julia kernel there exists also a [Ruby kernel](https://github.com/SciRuby/iruby) for IPython.
* [IRuby Demo Notebook](http://nbviewer.ipython.org/github/SciRuby/sciruby-notebooks/blob/master/getting_started.ipynb)
* [SciRuby Notebooks](https://github.com/SciRuby/sciruby-notebooks)
The interactive plotting library [Nyaplot](https://github.com/domitry/nyaplot) has some case studies using IRuby:
* [War expenditure per GDP](http://nbviewer.ipython.org/github/domitry/nyaplot/blob/master/examples/notebook/Mapnya.ipynb#Case2-:Fill-countries-in-different-colors)
* [Finding shape consensus among multiple geometrical polygons](http://nbviewer.ipython.org/gist/mgiraldo/a68b53175ce5892531bc)
### Perl
* An example showcasing full use of the [display protocol](http://nbviewer.ipython.org/github/zmughal/zmughal-iperl-notebooks/blob/master/IPerl-demos/20150209_IPerl_display_demo.ipynb) with the IPerl kernel.
## Miscellaneous topics about doing various things *with* the Notebook itself
* [Blogging With IPython in Blogger](http://nbviewer.ipython.org/github/fperez/blog/blob/master/120907-Blogging%20with%20the%20IPython%20Notebook.ipynb), also available in [blog post form](http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html), [full repo here](https://github.com/fperez/blog). By Fernando Perez.
* [Blogging With IPython in Octopress](http://nbviewer.ipython.org/urls/github.com/jakevdp/jakevdp.github.com/raw/master/downloads/notebooks/nb_in_octopress.ipynb), by Jake van der Plas and available as a [blog post](http://jakevdp.github.com/blog/2012/10/04/blogging-with-ipython/). Other [notebooks by Jake](https://github.com/jakevdp/jakevdp.github.com/tree/master/downloads/notebooks) contain many more great examples of doing interesting work with the scientific Python stack.
* [Blogging With IPython in Nikola](http://nbviewer.ipython.org/url/www.damian.oquanta.info/posts/blogging-with-nikola-and-ipython.ipynb), also available in [blog post form](http://www.damian.oquanta.info/posts/blogging-with-nikola-and-ipython.html) by Damián Avila.
* [Custom CSS control of the notebook](http://nbviewer.ipython.org/github/Carreau/posts/blob/master/Blog1.ipynb), this is part of a [blog repo](https://github.com/Carreau/posts) by Matthias Bussonnier.
* [IPython display hookery: tools to help display visual output from various sources](http://nbviewer.ipython.org/gist/5162445), a gist by [@deeplook](https://github.com/deeplook).
* [Importing IPython Notebooks as Modules](http://nbviewer.ipython.org/gist/6011986) by [Min RK](http://github.com/minrk).
## Reproducible academic publications
This section contains academic papers that have been published in the peer-reviewed literature or pre-print sites such as the [ArXiv](http://arxiv.org) that include one or more notebooks that enable (even if only partially) readers to reproduce the results of the publication. If you include a publication here, please link to the journal article as well as providing the nbviewer notebook link (and any other relevant resources associated with the paper).
1. [Reply to 'Influence of cosmic ray variability on the monsoon rainfall and temperature': a false-positive in the field of solar-terrestrial research](http://arxiv.org/abs/1502.00505) by [Benjamin Laken](http://www.benlaken.com), 2015. Reviewed article will appear in JASTP. The [IPython notebook](http://nbviewer.ipython.org/github/benlaken/Comment_BadruddinAslam2014/blob/master/Monsoon_analysis.ipynb) reproduces the full analysis and figures exactly as they appear in the article, and is available on Github: link via [figshare](http://figshare.com/articles/Comment_on_Badruddin_amp_Aslam_2014_/1299413).
1. [The probability of improvement in Fisher's geometric model: a probabilistic approach](http://dx.doi.org/10.1016/j.tpb.2014.10.004), by [Yoav Ram](http://www.yoavram.com/) and [Lilach Hadany](https://sites.google.com/site/hadanylab/). (Theoretical Population Biology, 2014). An [IPython notebook](http://nbviewer.ipython.org/url/www.sciencedirect.com/science/MiamiMultiMediaURL/1-s2.0-S0040580914000811/1-s2.0-S0040580914000811-mmc1.txt/272364/FULL/S0040580914000811/471cf02085a52c248dc76ae65ad4409d/mmc1.txt), allowing figure reproduction, was deposited as a [supplementry file](http://www.sciencedirect.com/science/MiamiMultiMediaURL/1-s2.0-S0040580914000811/1-s2.0-S0040580914000811-mmc1.txt/272364/FULL/S0040580914000811/471cf02085a52c248dc76ae65ad4409d/mmc1.txt).
1. [Stress-induced mutagenesis and complex adaptation](http://rspb.royalsocietypublishing.org/content/281/1792/20141025.abstract), by [Yoav Ram](http://www.yoavram.com/) and [Lilach Hadany](https://sites.google.com/site/hadanylab/) (Proceedings B, 2014). An [IPython notebook](https://github.com/yoavram/ruggedsim/blob/master/manuscript/supplementry.ipynb), allowing figures reproduction, was deposited as a [supplementry file](http://rspb.royalsocietypublishing.org/content/suppl/2014/08/19/rspb.2014.1025.DC1).
1. [Automatic segmentation of odor maps in the mouse olfactory bulb using regularized non-negative matrix factorization](http://www.sciencedirect.com/science/article/pii/S1053811914003103), by J. Soelter et al. (Neuroimage 2014, Open Access). The [notebook](http://nbviewer.ipython.org/github/jansoe/FUImaging/blob/master/examples/IOSsegmentation/regNMF.ipynb) allows to reproduce most figures from the paper and provides a deeper look at the data. The [full code repository](https://github.com/jansoe/FUImaging/tree/Neuroimage2014) is also available.
1. [Multi-tiered genomic analysis of head and neck cancer ties TP53 mutation to 3p loss, by A. Gross et al. (Nature Genetics 2014)](http://www.nature.com/ng/journal/vaop/ncurrent/full/ng.3051.html). The [full collection of notebooks to replicate the results](https://github.com/theandygross/TCGA/tree/master/Analysis_Notebooks#guide-to-running).
1. [powerlaw: a Python package for analysis of heavy-tailed distributions, by J. Alstott et al.](https://code.google.com/p/powerlaw/). [Notebook of examples in manuscript](http://nbviewer.ipython.org/gist/19fcdd6a4ba400ce8de2), [ArXiv link](http://arxiv.org/abs/1305.0215) and [project repository](https://github.com/jeffalstott/powerlaw).
1. [Collaborative cloud-enabled tools allow rapid, reproducible biological insights, by B. Ragan-Kelley et al.](http://www.nature.com/ismej/journal/v7/n3/full/ismej2012123a.html). The [main notebook](http://nbviewer.ipython.org/gist/3693491/cloud_demo_complete.ipynb), the [full collection of related notebooks](http://nbviewer.ipython.org/gist/3693491) and the [companion site](http://qiime.org/home_static/nih-cloud-apr2012) with the Amazon AMI information for reproducing the full paper.
1. [A Reference-Free Algorithm for Computational Normalization of Shotgun Sequencing Data, by C.T. Brown et al.](http://ged.msu.edu/papers/2012-diginorm). [Full notebook](http://nbviewer.ipython.org/urls/github.com/ged-lab/2012-paper-diginorm/raw/master/notebook/diginorm.ipynb), [ArXiv link](http://arxiv.org/abs/1203.4802) and [project repository](https://github.com/ged-lab/2012-paper-diginorm).
1. [The kinematics of the Local Group in a cosmological context](http://arxiv.org/abs/1303.2690) by [J.E. Forero-Romero et al.](http://wwwprof.uniandes.edu.co/~je.forero/). The [Full notebook](http://nbviewer.ipython.org/github/forero/LG_Kinematics/blob/master/code/main_analysis.ipynb) and also all the data in a [github repo](https://github.com/forero/LG_Kinematics).
1. [Warming Ocean Threatens Sea Life](http://www.scientificamerican.com/article.cfm?id=warming-ocean-threatens-sea-life), an article in Scientific American [backed by a notebook for its main plot](http://nbviewer.ipython.org/github/robertodealmeida/notebooks/blob/master/scientific_american/Scientific%20American%20graph.ipynb). By [Roberto de Almeida](https://github.com/robertodealmeida) from [MarinExplore](https://marinexplore.com).
1. [Extrapolating Weak Selection in Evolutionary Games](http://nbviewer.ipython.org/github/juliangarcia/ews/blob/master/notebook.ipynb), by Wu, García, Hauert and Traulsen. [PLOS Comp Bio paper](http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1003381) and [Figshare link](http://figshare.com/articles/Extrapolating_weak_selection_in_evolutionary_games_source_code/814470).
1. [Using neural networks to estimate redshift distributions. An application to CFHTLenS]
(http://nbviewer.ipython.org/urls/bitbucket.org/christopher_bonnett/nn_notebook/raw/5e69b55193a229cb2076a2f18e43b45c56e317e0/T-800.ipynb)
by Christopher Bonnett [paper](http://arxiv.org/abs/1312.1287)(submitted to MNRAS)
1. [Mechanisms for stable, robust, and adaptive development of orientation maps in the primary visual cortex](http://dx.doi.org/10.1523/JNEUROSCI.1037-13.2013) by Jean-Luc R. Stevens, Judith S. Law, Jan Antolik, and James A. Bednar. Journal of Neuroscience, 33:15747-15766, 2013. [Notebook1]
(https://ioam.github.io/topographica/_static/gcal_notebook.html), [Notebook2](https://ioam.github.io/topographica/_static/stevens_jn13_notebook.html).
1. [Accelerated Randomized Benchmarking](http://nbviewer.ipython.org/github/cgranade/accelerated-randomized-benchmarking/blob/master/src/model_testing.ipynb), by [Christopher Granade](http://www.cgranade.com/), [Christopher Ferrie](https://sites.google.com/site/csferrie/) and D. G. Cory. [New Journal of Physics **17** 013042 (2015)](http://iopscience.iop.org/article/10.1088/1367-2630/17/1/013042/meta;jsessionid=1F48CDD7C7D7849B0777C495ED1551CC.c1), [arXiv](http://arxiv.org/abs/1404.5275), [GitHub repo](https://github.com/cgranade/accelerated-randomized-benchmarking).
1. [Dynamics and associations of microbial community types across the human body](http://dx.doi.org/10.1038/nature13178), by Tao Ding & Patrick D. Schloss. [Notebook replicating results](http://nbviewer.ipython.org/gist/pschloss/9815766/notebook.ipynb).
1. [Variations in submarine channel sinuosity as a function of latitude and slope](http://nbviewer.ipython.org/gist/zsylvester/6040d0015b9b907bc788), by Sylvester, Z., Pirmez, C., Cantelli, A., & Jobe, Z. R.
1. [Frontoparietal representations of task context support the flexible control of goal directed cognition](http://www.jneurosci.org/content/34/32/10743.short), by M.L. Waskom, D. Kumaran, A.M. Gordon, J. Rissman, & A.D. Wagner. [Github repository](https://github.com/WagnerLabPapers/Waskom_JNeurosci_2014) | [Main notebook](http://nbviewer.ipython.org/github/WagnerLabPapers/Waskom_JNeurosci_2014/blob/master/Behavioral_and_Decoding_Analyses.ipynb)
1. [pyparty: Intuitive Particle Processing in Python](http://openresearchsoftware.metajnl.com/article/view/jors.bh), Adam Hughes [Notebook to Generate the Published Figures](http://nbviewer.ipython.org/github/hugadams/pyparty/blob/master/examples/Notebooks/JORS_data.ipynb?create=1) | Also, check out the [pyparty tutorial notebooks](https://github.com/hugadams/pyparty).
1. [Indication of family-specific DNA methylation patterns in developing oysters](http://biorxiv.org/content/early/2014/12/16/012831), Claire E. Olson, Steven B. Roberts
doi: http://dx.doi.org/10.1101/012831. [Notebook to generate results in the paper](http://nbviewer.ipython.org/github/che625/olson-ms-nb/blob/master/BiGo_dev.ipynb).
1. [Parallel Prefix Polymorphism Permits Parallelization, Presentation & Proof](http://conferences.computer.org/hptcdl/2014/papers/7020a047.pdf), [Jiahao Chen](http://jiahao.github.io) and [Alan Edelman](http://www-math.mit.edu/~edelman/), HPTCDL'14. [Website](http://jiahao.github.io/parallel-prefix) and [notebook](https://github.com/jiahao/ijulia-notebooks/blob/master/2014-08-06-parallel-prefix.ipynb)
1. [Transcriptome Sequencing Reveals Potential Mechanism of Cryptic 3 Splice Site Selection in *SF3B1*-mutated Cancers](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004105) by [Christopher DeBoever](http://cdeboever3.github.io/) et al. There are several [notebooks](https://github.com/cdeboever3/deboever-sf3b1-2015/tree/master/notebooks) to replicate results and make figures.
1. [A Workflow for Characterizing Nanoparticle Monolayers for Biosensors: Machine Learning on Real and Artificial SEM Images](https://peerj.com/preprints/671/), Adam Hughes, Zhaowen Liu, Maryam Raftari, Mark. E Reeves. Notebooks are linked in Table 1 in the text.
1. [AtomPy: An Open Atomic Data Curation Environment for Astrophysical Applications](http://www.mdpi.com/2218-2004/2/2/123), by C. Mendoza, J. Boswell, D. Ajoku, M. Bautista.
1. [Visualizing 4-Dimensional Asteroids](http://blogs.scientificamerican.com/sa-visual/2014/09/16/visualizing-4-dimensional-asteroids/), in Scientific American (by Jake VanderPlas)
1. [Challenges and opportunities in understanding microbial communities with metagenome assembly](http://journal.frontiersin.org/article/10.3389/fmicb.2015.00678/abstract), [accompanied by IPython Notebook tutorial](http://nbviewer.ipython.org/github/germs-lab/frontiers-review-2015/blob/master/frontiers-nb-2015.ipynb), by [Adina Howe](http://germslab.org) and Patrick Chain.
## Data-driven journalism
* [The Need for Openness in Data Journalism](http://nbviewer.ipython.org/github/brianckeegan/Bechdel/blob/master/Bechdel_test.ipynb), by [Brian Keegan](http://www.brianckeegan.com).
* [St. Louis County Segregation Analysis](https://github.com/BuzzFeedNews/2014-08-st-louis-county-segregation) , analysis for the article [The Ferguson Area Is Even More Segregated Than You Probably Guessed](http://www.buzzfeed.com/jsvine/the-ferguson-area-is-even-more-segregated-than-you-thought) by [Jeremy Singer-Vine](https://twitter.com/jsvine).
## Whimsical notebooks
* [XKCD-styled plots created with Matplotlib](http://nbviewer.ipython.org/url/jakevdp.github.com/downloads/notebooks/XKCD_plots.ipynb). Here is the [blog post version](http://jakevdp.github.com/blog/2012/10/07/xkcd-style-plots-in-matplotlib/) with discussion. By Jake van der Plas.
* [Van Gogh's Starry Night with ipythonblocks](http://nbviewer.ipython.org/github/jiffyclub/ipythonblocks/blob/master/demos/starry_night_to_text.ipynb), part of Matt Davis' [ipythonblocks](https://github.com/jiffyclub/ipythonblocks). This is a teaching tool for use with the IPython notebook that provides visual elements to understand programming concepts.
* [Conway's Game of Life](http://nbviewer.ipython.org/gist/3778422). Interesting use of convolution operation to calculate the next state of game board, instead of obvious find neighbors and filter the board for next state.
* [pynguins](https://gist.github.com/denfromufa/9a5e1fdeaf611dc60ea8). Using jupyter notebook, python, and numpy to solve Board Game "Penguins on Ice".
* ["People plots"](http://nbviewer.ipython.org/gist/4544012), stick figures generated with matplotlib.
* [Reveal converter mini-tutorial](http://nbviewer.ipython.org/url/www.damian.oquanta.info/posts/reveal-converter-mini-tutorial.ipynb), also available in [blog post form](http://www.damian.oquanta.info/posts/reveal-converter-mini-tutorial.html). Do you want to make static html/css slideshow straight from the IPython notebook? OK, now you can do it with the *reveal* converter (nbconvert). [Demo](http://www.slideviper.oquanta.info/tutorial/slideshow_tutorial_slides.html) by Damián Avila.
* [Personal IPython Weight Notebook] (http://nbviewer.ipython.org/gist/9769238). Plot your loss of weight with prognosis and motivation features.
* [Streaming Double Pendulum Simulation in IPython NB](http://nbviewer.ipython.org/github/plotly/python-user-guide/blob/master/s7_streaming/s7_streaming.ipynb).
* [Porque Charles Xavier debe cambiar a Cerebro por Python](http://nbviewer.ipython.org/github/mshopper/aurora/blob/master/Aurora.ipynb), a study in data and gender in the Marvel comics universe, by [Mai Giménez](http://twitter.com/adahopper) and [Angela Rivera](http://twitter.com/ghilbrae).
* [Functional Geometry: a deconstruction of the MC Escher woodcut Square Limit](http://nbviewer.ipython.org/github/shashi/ijulia-notebooks/blob/master/funcgeo/Functional%20Geometry.ipynb), an IJulia notebook by [Shashi Gowda](http://shashi.github.io/).
## Videos of IPython being used in the wild
Of course the first thing you might try is searching for videos about IPython (1900 or so by last count on Youtube) but there are demonstrations of other applications using the power of IPython but are not mentioned is the descriptions. Below are a few such:
* [Video](http://www.youtube.com/watch?v=Nc16qeGBtMU) on how to learn Python featuring IPython as the platform of choice for learning!
* [This video](http://www.youtube.com/watch?v=4ONBVNm3isI) shows IPython being used in the [scikit-learn](http://scikit-learn.org) project
* He doesn't show IPython in use but his IPython sticker is clear for the entire video: [Planning and Tending the Garden: The Future of Early Childhood Python Education](http://www.youtube.com/watch?v=op61s-QHryk)
* [Wes McKinney's speech](http://youtu.be/qbYYamU42Sw?t=5m9s) on Python and data analysis features IPython as does his book [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do)
* [This video](https://www.youtube.com/watch?v=zG8FYPFU9n4) shows Plotly and IPython in use at a Montreal Python meetup.

111
userguide/en/notebook/nb.md Normal file
View File

@ -0,0 +1,111 @@
## Notebook Usage ##
Open Dashboard, create Python3 Notebook by selecting ```New - Python3```,
will then enter Python3 Notebook interactive programming environment.
At the same time, an IPython Notebook file named `untitled.ipynb` is
generated for this interaction.
This section will introduce a basic [Hello World](#hello-world),
a [Visualizing Machine learning result](#visualizing-machine-learning-result),
a [complete tutorial](#complete-tutorial).
### Hello World ###
In a new Cell, input Python3 code:
```python
print("Hello World")
```
Run and get the result:
```
Hello World
```
### Visualizing Machine Learning Result
Visualizing need the support of `matplotlib`. It should be loaded first.
Input and execute the following code in a new Cell:
```python
%matplotlib inline
```
or
```python
%matplotlib notebook
```
If no error, the browser is now ready to draw graphs.
Input and execute the sample machine learnign code in a new Cell:
```python
# import package
import numpy as np
import matplotlib.pyplot as plt
from sklearn import linear_model, datasets
# load data : we only use target==0 and target==1 (2 types classify) and feature 0 and feature 2 ()
iris = datasets.load_iris()
X = iris.data[iris.target!=2][:, [0,2]]
Y = iris.target[iris.target!=2]
h = .02 # step size in the mesh
logreg = linear_model.LogisticRegression(C=1e5)
logreg.fit(X, Y)
# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, m_max]x[y_min, y_max].
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
#plt.figure(1, figsize=(4, 3))
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=Y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())
plt.savefig("learn.svg")
```
Later on, you will see the output like:
<img src="../images/sklearn1.png">
At the same time, a vector graph ```learn.svg``` is generated in the
current directory, which can be opened in the Dashboard.
You can also open it directly here by input and execute
```
SVG("learn.svg")
```
### Complete Tutorial ###
Many tasks can be done in IPython Notebook, some are interesting, for example, displaying kinds of JPG/PNG/SVG pictures, videos, HTML files, pdf files, loading external resources like an remote WEB page, even a Youtube video, showing LaTeX math formula, etc.
The complete user guide and tutorials can be refered
[here](http://nbviewer.jupyter.org/github/ipython/ipython/blob/4.0.x/examples/IPython%20Kernel/Index.ipynb).
[Jupyter nbviewer](http://nbviewer.jupyter.org) also shows many examples.

View File

@ -0,0 +1,821 @@
## Python Packages ##
Python3 packages in the Docklet base image :
<table>
<caption></caption>
<thead>
<tr>
<th>Python Software</th>
<th>License</th>
<th>Summary</th>
<th class="center-txt">Note</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="middle">
<a target="_blank" href="http://matplotlib.org/basemap/">basemap</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://matplotlib.sourceforge.net/users/license.html">PSF</a></td>
<td valign="middle">Plot data on map projections with Matplotlib</td>
<td class="center-txt">
Map
</td>
</tr><tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/Blosc/bcolz">bcolz</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt">BSD</a></td>
<td valign="middle">Columnar and compressed data containers.</td>
<td class="center-txt">
Data
</td>
</tr><tr>
<td valign="middle">
<a target="_blank" href="https://github.com/pyca/bcrypt">bcrypt</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://pypi.python.org/pypi/bcrypt/">Apache</a></td>
<td valign="middle">Modern password hashing for your software and your servers</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.crummy.com/software/BeautifulSoup/">BeautifulSoup4</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.crummy.com/software/BeautifulSoup/">MIT</a></td>
<td valign="middle">Sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://biopython.org/wiki/Main_Page">biopython</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.biopython.org/DIST/LICENSE">(as-is) BSD-like</a></td>
<td valign="middle">Tools for biological computation</td>
<td class="center-txt">
Biological
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/ContinuumIO/blaze">blaze</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Data migration utilities</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://bokeh.pydata.org/">bokeh</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Statistical and novel interactive HTML plots for Python</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://scitools.org.uk/cartopy/docs/latest/index.html">cartopy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://scitools.org.uk/cartopy/docs/latest/index.html">LGPL</a></td>
<td valign="middle">A Python package designed to make drawing maps for data analysis and visualisation as easy as possible</td>
<td class="center-txt">
Map
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/enthought/chaco">chaco</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/enthought/chaco/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Library for generating interactive 2D plots</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://www.cheetahtemplate.org/">cheetah</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Template engine and code generation tool</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://nedbatchelder.com/code/modules/coverage.html">coverage</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Code coverage measurement for Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/cryptography">cryptography</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">cryptography is a package which provides cryptographic recipes and primitives to Python developers.</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://cython.org/">Cython</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/cython/cython/blob/master/LICENSE.txt">Apache</a></td>
<td valign="middle">Python-based language for writing C extensions for Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://docutils.sourceforge.net">docutils</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://svn.python.org/projects/external/docutils-0.5/COPYING.txt">public-domain with exceptions (BSD, PSF)</a></td>
<td valign="middle">Documentation utilities</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/flask">flask</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A lightweight web application framework based on Werkzeug, Jinja2 and good intentions.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://github.com/mitsuhiko/flask-babel">Flask-Babel</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://github.com/mitsuhiko/flask-babel">BSD</a></td>
<td valign="middle">Adds i18n/l10n support to Flask applications</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.gevent.org">gevent</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Python network library using greenlet and libevent for easy and scalable concurrency</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://bitbucket.org/Jeffrey/gevent-websocket">gevent_websocket</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a></td>
<td valign="middle">Websocket handler for the gevent pywsgi server, a Python network library</td>
<td class="center-txt">
Network
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/yhat/ggplot/">ggplot</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/yhat/ggplot/">BSD</a></td>
<td valign="middle">ggplot for python</td>
<td class="center-txt">
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.h5py.org/">h5py</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.h5py.org/docs/meta/licenses.html">BSD</a></td>
<td valign="middle">Python interface to the HDF library</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://jinja.pocoo.org/2/">Jinja2</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://dev.pocoo.org/hg/jinja2-main/raw-file/744e087d24e9/LICENSE">BSD</a></td>
<td valign="middle">Template engine</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://jupyter.org">jupyter</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://jupyter.org">BSD</a></td>
<td valign="middle">Jupyter metapackage. Install all the Jupyter components in one go.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://lmfit.github.io/lmfit-py/">lmfit</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://lmfit.github.io/lmfit-py/">BSD</a></td>
<td valign="middle">Least-Squares Minimization with Bounds and Constraints</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://lxml.de/">lxml</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://lxml.de/index.html#license">BSD</a></td>
<td valign="middle">XML/XSLT library with bindings to libxml2/libxslt</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/M2Crypto">M2Crypto</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">M2Crypto: A Python crypto and SSL toolkit</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://matplotlib.sourceforge.net/">matplotlib</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://matplotlib.org/users/license.html">PSF</a></td>
<td valign="middle">Interactive 2D plotting library</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://mdp-toolkit.sourceforge.net/">MDP</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://mdp-toolkit.sourceforge.net/install.html">BSD</a></td>
<td valign="middle">Modular toolkit for Data Processing (MDP)</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/memory_profiler">memory_profiler</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://pypi.python.org/pypi/memory_profiler">BSD</a></td>
<td valign="middle">A module for monitoring memory usage of a python program</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://msgpack.org/">msgpack</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Efficient binary serialization library</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/mrocklin/multipledispatch/">multipledispatch</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/mrocklin/multipledispatch/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Multiple dispatch in Python.</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://networkx.lanl.gov/">networkx</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://networkx.lanl.gov/reference/legal.html">BSD</a></td>
<td valign="middle">Create, manipulate, and analyze graphs and networks</td>
<td class="center-txt">
Graph
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.nltk.org/">nltk</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Suite of tools for symbolic and statistical natural language processing</td>
<td class="center-txt">
NLP
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/numexpr">numexpr</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Fast evaluation of array expressions</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://numpy.scipy.org/">numpy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">General-purpose multi-dimensional array-processing and math library</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://bitbucket.org/ericgazoni/openpyxl/wiki/Home">openpyxl</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT/Expat</a></td>
<td valign="middle">Read and and write Excel OpenXML files</td>
<td class="center-txt">
Excel
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pandas.pydata.org/">pandas</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Data manipulation and analysis library</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/Paste">paste</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Tools for using a Web Server Gateway Interface stack</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/patsy">patsy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Package for describing statistical models and building design matrices</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pep8/">pep8</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://code.enthought.com/license.php">BSD</a></td>
<td valign="middle">Python style guide checker</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pexpect.readthedocs.org/">pexpect</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://pexpect.readthedocs.org/">ISC license</a></td>
<td valign="middle">Pexpect allows easy control of interactive console applications.</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.pythonware.com/products/pil/">PIL</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.pythonware.com/products/pil/license.htm">MIT</a></td>
<td valign="middle">Image processing library</td>
<td class="center-txt">
Image
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/plotly">plotly</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Python plotting library for collaborative, interactive, publication-quality graphs.</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.dabeaz.com/ply/">ply</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.dabeaz.com/ply/">BSD</a></td>
<td valign="middle">Python implementation of lex and yacc</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://olgabot.github.io/prettyplotlib">prettyplotlib</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/olgabot/prettyplotlib/blob/master/LICENSE.txt">MIT</a></td>
<td valign="middle">Painlessly create beautiful default matplotlib plots.</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://code.google.com/p/psutil/">psutil</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">OS interface to processes and tasks</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/py">py</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Library with cross-python path, ini-parsing, io, code, log facilities</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://rhodesmill.org/pyephem/">pyephem</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.gnu.org/licenses/lgpl.txt">LGPL</a></td>
<td valign="middle">Scientific-grade astronomy routines</td>
<td class="center-txt">
Astronomy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pyflakes">pyflakes</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Static analysis of Python code to detect common errors</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pygments.org/">Pygments</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://bitbucket.org/birkenfeld/pygments-main/src/fbeebf94f4d2/LICENSE">BSD</a></td>
<td valign="middle">Code syntax highlighting package written in Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pyOpenSSL">pyOpenSSL</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/LICENSE">Apache License 2</a></td>
<td valign="middle">Python bindings to the OpenSSL library</td>
<td class="center-txt">
Network
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/PySal">pysal</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A library of spatial analysis functions.</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.pytables.org/">PyTables</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.pytables.org/moin/License">BSD</a></td>
<td valign="middle">Hierarchical datasets for extremely large data</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pytest.org/">pytest</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Simple powerful testing with Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://pyyaml.org/wiki/PyYAML">PyYAML</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">YAML parser and emitter</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://github.com/ellisonbg/pyzmq">pyzmq</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/zeromq/pyzmq/blob/master/COPYING.BSD">BSD-like</a></td>
<td valign="middle">Binding to the ZeroMQ fast messaging library</td>
<td class="center-txt">
Middleware
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://docs.python-requests.org/en/latest/">requests</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Elegant and simple HTTP library for Python</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://scikit-learn.sourceforge.net/">scikit_learn</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Machine learning and data mining routines</td>
<td class="center-txt">
ML
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.scipy.org/">SciPy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/scipy/scipy/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Libraries for mathematics, science, and engineering</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://www.scons.org">SCons</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Pythonic substitute for Make</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://stanford.edu/~mwaskom/software/seaborn/">seaborn</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Seaborn: statistical data visualization</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://packages.python.org/setuptools/">setuptools</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.python.org/psf/license/">PSF</a></td>
<td valign="middle">Easily download, build, install, upgrade, and uninstall Python packages</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/Toblerity/Shapely">Shapely</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/Toblerity/Shapely/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Geometric objects, predicates, and operations</td>
<td class="center-txt">
Geo
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://cheeseshop.python.org/pypi/simplegeneric">simplegeneric</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://cheeseshop.python.org/pypi/simplegeneric">ZPL 2.1</a></td>
<td valign="middle">Simple generic functions (similar to Python's own len(), pickle.dump(), etc.)</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://sphinx-doc.org/">Sphinx</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Creates intelligent and beautiful project documentation</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.SQLAlchemy.org/">SQLAlchemy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">SQL toolkit and Object Relational Mapper</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://statsmodels.sourceforge.net/">statsmodels</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/statsmodels/statsmodels/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Statistical computations and models for use with SciPy</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://code.google.com/p/sympy/">SymPy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Symbolic mathematics library</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.tornadoweb.org/">Tornado</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a></td>
<td valign="middle">Scalable, non-blocking web server</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://virtualenv.pypa.io/">virtualenv</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://virtualenv.pypa.io/">MIT</a></td>
<td valign="middle">Virtual Python Environment builder</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/werkzeug">werkzeug</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Advanced WSGI utility modules</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/xarray">xarray</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/pydata/xarray">Apache</a></td>
<td valign="middle">N-D labeled arrays and datasets in Python</td>
<td class="center-txt">
Math
</td>
</tr><tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.python-excel.org/">xlrd</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/python-excel/xlrd/blob/master/xlrd/licences.py">BSD</a></td>
<td valign="middle">Extract data from Microsoft Excel (tm) spreadsheet files</td>
<td class="center-txt">
Excel
</td>
</tr><tr>
<td valign="middle">
<a target="_blank" href="http://xlsxwriter.readthedocs.org/">xlsxwriter</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A Python module for creating Excel XLSX files.</td>
<td class="center-txt">
Excel
</td>
</tr>
</tbody>
<tfoot></tfoot>
</table>

1845
userguide/en/notebook/r.md Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,10 @@
## Docklet Portal ##
The web pages of Docklet portal include:
* [Dashboard](dashboard.md)
* [Config](config.md)
* [Status](status.md)
* [Hosts](hosts.md)
* [Users](users.md)
* [Admin](admin.md)

View File

@ -0,0 +1,4 @@
## Admin ##
This webpage requires *admin* role to access.

View File

@ -0,0 +1,22 @@
## Config ##
Users will configure their Workspace here, mainly the management of
vclusters and images.
The following picture shows the Config webpage of **tee** Workspace:
<img src="../images/config-workspace.png" width="600" alt="workspace
config">
Users can add nodes to or remove nodes from the vcluster. The above
picture show the *tee* Workspace having two nodes, with the detailed
information about node's name, IP address, status and image created
from, etc.
Click **Save** to save the Workspace as a image named *myimage*, which
can be used later on. Now the image list is shown as:
<img src="../images/config-image.png" width="700" alt="image config">
Users can also **Share** their saved private images with others, to help
them avoid tedious installation and configuration stuff.

View File

@ -0,0 +1,54 @@
## Dashboard ##
Dashboard is where users manage their Workspaces. Users can view
, create, start, stop, delete and enter their Workspaces.
Initially the Workspace list is empty. Users need to create a Workspace
first. Click on **Add Workspace** will enter the web page for creating
Workspace, then select the image, and create a Workspace named **tee**, as
illustrated as follows:
<img src="../images/workspace-create.png" width="600" alt="add
workspace">
If the creation succeed, the Workspace list changed to:
<img src="../images/dashboard-stop.png" width="600" alt="dashboard
stopped status">
We can see a Workspace named **tee** has been created, with status **Stopped**.
Now the user can choose to **Start** or **Delete** the Workspace.
The *Stopped* Workspace still exists and wait for next start. Users are
encouraged to stop their Workspace when not running jobs to reduce
resource consumption.
Now click **Start** to start the Workspace. The Workspace status changed
to:
<img src="../images/dashboard-start.png" width="600" alt="dashboard
running status">
The Workspace status is **Running**. Users can choose to **Stop** the
Workspace, or **Go** into the Workspace. They can also click the name
*tee* to [Config](config.md) the Workspace. If click on the status
**Running**, will show the detailed vcluster [Status](status.md).
**Note**:
1. If the users click **Go** soon after creation of a Workspace,
they may get an error message of `503: Proxy Target Missing`, this is possibly
because of the Workspace backend initialization not finished. Click
**Go** some time later, users will enter the Workspace as expected.
2. If the users click **Go**, but get `Not Found` error message:
```
Not Found
The requested URL was not found on the server. If you entered the URL
manually please check your spelling and try again.
```
It is possibily because the system has experiened a recovery. Users need to
manually reboot their Workspaces to gain access.

View File

@ -0,0 +1,23 @@
## Hosts ##
This webpage requires *admin* role to access.
The Hosts webpage shows the status of the Docklet physical cluster,
including host IP address, status, containers hosted, resource
usage, etc.
The following show the information of a demo Docklet physical cluster:
<img src="../images/hosts.png" width="600" alt="hosts">
Click the host's **Realtime** button, will get the configuration
information of the host:
<img src="../images/hosts-summary-config.png" width="600" alt="
summary config of one host">
Also the host's realtime resource usage activity:
<img src="../images/hosts-summary-perf.png" width="600" alt="
realtime performance of one host">

View File

@ -0,0 +1,13 @@
### Status ###
The Status web page shows the vcluster monitoring information. Here users can see the load info of their vclusters, including CPU and memory activity.
The following picture show the general load of the *tee* Workspace
vcluster:
<img src="../images/status.png" width="600" alt="vcluster status">
Click **Realtime**, will get the realtime usage of the node's CPU and
memory, shown as follows:
<img src="../images/status-summary.png" width="600" alt="vcluster status summary of one node">

View File

@ -0,0 +1,24 @@
## Users ##
This webpage requires *admin* role to access.
Through it administators can get and modify users' information,
and add user or usergroup at the same time.
### Register & Activating ###
Now docklet account cannot be registered by users, it uses external authenticating system like pam to let users log in.
Docklet account need activating to use some specific functions.When an external account is used at the first time, a notification will be shown in the upper left corner of the webpage, which says you should hand in a request form.
<img src="../images/user_init.png" width="300" alt="add workspace">
By clicking it, you can enter the activating page. Your E-mail, student/staff ID, department, real name and reason to use Docklet is required.After handing in the request, Docklet administrators will verify it in 3 days.
Your browser will log out Docklet automatically, re-login is required to refresh user info.Before your request is verified, a notification will be shown in upper left corner.It can be closed by clicking.
## 试用 ##
docklet支持用户以访客模式试用在你的帐号审核通过之前你可以通过点击主页的have a try按钮进入
试用界面在试用模式下你可以点击Go按钮进入到一个workspace中。

View File

@ -0,0 +1,10 @@
# Development in WEB Terminal #
* [Available Packages](list.md)
* [C/C++ Example](c.md)
* [Java Example](java.md)
* [MPI Example](mpi.md)
* [Spark Example](spark.md)
* [LaTeX Example](latex.md)

171
userguide/en/software/c.md Normal file
View File

@ -0,0 +1,171 @@
## C/C++ Example##
Docklet provides GCC 5+ and GDB to help develop C program.
### C
Below is the source code of file `sum.c`:
```c
#include <stdio.h>
#define TOP 50
int main()
{
int i = 0 , sum = 0 ;
while ( i <= TOP ) {
sum += i ;
i++ ;
}
printf(" Sum of 1 to %d is %d\n", TOP, sum ) ;
return 0 ;
}
```
Open WEB Terminal, compile and run
```
$ gcc -g sum.c -o sum
$ ./sum
Sum of 1 to 50 is 1275
```
### C++ ###
The following is a C++ program with bugs from [here](http://www.cprogramming.com/gdb.html).
The buggy `main.cpp`:
```c++
#include<iostream>
using namespace std;
long factorial(int n);
int main()
{
int n(0);
cin>>n;
long val=factorial(n);
cout<<val;
cin.get();
return 0;
}
long factorial(int n)
{
long result(1);
while(n--)
{
result*=n;
}
return result;
}
```
Open WEB Terminal, compile and run: input 5 and get wrong result 0.
```
$ g++ -g main.cpp -o main
$ ./main
5
0
```
Now debug with gdb
```
$ gdb ./main
1. $ g++ main.cpp -g -Wall -o main
2. $ gdb main
3. GNU gdb (Ubuntu 7.10.1-0ubuntu1) 7.10.1
4. Copyright (C) 2015 Free Software Foundation, Inc.
5. This GDB was configured as "i686-redhat-linux-gnu".
6. For bug reporting instructions, please see:
7. <http://www.gnu.org/software/gdb/bugs/>...
8. Reading symbols from /root/main...done.
9. (gdb) break 11
10. Breakpoint 1 at 0x80485f9: file main.cpp, line 11.
11. (gdb) run
12. Starting program: /root/main
13. 3
14.
15. Breakpoint 1, main () at main.cpp:11
16. 11 long val=factorial(n);
17. (gdb) step
18. factorial (n=3) at main.cpp:19
19. 19 long result(1);
20. (gdb) list
21. 14 return 0;
22. 15 }
23. 16
24. 17 long factorial(int n)
25. 18 {
26. 19 long result(1);
27. 20 while(n--)
28. 21 {
29. 22 result*=n;
30. 23 }
31. (gdb) watch n
32. Hardware watchpoint 2: n
33. (gdb) watch result
34. Hardware watchpoint 3: result
35. (gdb) continue
36. Continuing.
37. Hardware watchpoint 3: result
38.
39. Old value = 0
40. New value = 1
```
Var `result` is initialized to 1.
```
41. factorial (n=3) at main.cpp:20
42. 20 while(n--)
43. (gdb)
```
Note that no commands are put here, just <return> is hit.
It re-executes the last command.
```
44. Continuing.
45. Hardware watchpoint 2: n
46.
47. Old value = 3
48. New value = 2
```
Now `n` is immediately decremented from 3 to 2.
```
49. 0x08048654 in factorial (n=2) at main.cpp:20
50. 20 while(n--)
51. (gdb)
52. Continuing.
53. Hardware watchpoint 3: result
54.
55. Old value = 1
56. New value = 2
```
Now result becomes 2 (by multiplying result's earlier value with n's
value). We've found the first bug! result is supposed to be evaluated by
multiplying 3 * 2 * 1 but here the multiplication starts from 2. To
correct it, we have to change the loop a bit.
```c++
while(n>0) //doesn't let n reach 0
{
result*=n;
n--; //decrements only after the evaluation
}
```
After applying the fix, the test is passed.
[Here](http://www.cprogramming.com/gdb.html) describes the full
debugging process.
For how to use GDB, please reference the official
[GDB manual](http://sourceware.org/gdb/current/onlinedocs/gdb/)。

View File

@ -0,0 +1,33 @@
## Java Example ##
Docklet provides OpenJDK 7+ to develop Java program.
`NumberFactorial.java` is to calculate factorial of `n`:
```java
public class NumberFactorial {
public static void main(String[] args) {
int number = 5;
/*
* Factorial of any number is !n.
* For example, factorial of 4 is 4*3*2*1.
*/
int factorial = number;
for(int i =(number - 1); i > 1; i--) {
factorial = factorial * i;
}
System.out.println("Factorial of a number is " + factorial);
}
}
```
Open WEB Terminal, compile and run
```
$ javac NumberFactorial.java
$ java NumberFactorial
Factorial of a number is 120
```

View File

@ -0,0 +1,37 @@
## LaTeX Example ##
Docklet provides the full [TeX Live 2015](http://www.tug.org/texlive).
Users can edit and compile LaTeX source files, generating pdf files in
WEB Terminal. Then open the resulting pdf file in Jupyter Dashboard to
check the output.
### Hello World
A very simple `hello.tex` :
```latex
\documentclass{article}
\title{My \LaTeX ~Hello World}
\author{Zhang San}
\date{Feb 20, 2016}
\begin{document}
\maketitle
Hello World!
\end{document}
```
Compile using `pdflatex` or `xelatex`:
```
$ xelatex hello
$ ls
hello.aux hello.log hello.pdf hello.tex
```
### CJK Char Support ###
In TeX Live 2015, `xeCJK` + `xelatex` provides very good support for CJK and any other
non-ASCII chars. For CJK users, they only need to use the `xeCJK` package in their source file,
and then comile using `xelatex`, which will automatially use the
available truetype fonts in the system path, usually `/usr/share/fonts`. No extra work needed.

View File

@ -0,0 +1,38 @@
## Software Packages ##
The Docklet public base image has the following software:
** System **
* vim emcas nano
* tmux w3m curl
* zsh
* unrar,p7zip-full,p7zip-rar
* openssh-server
* openbox
* vpnc ( web )
* fonts-droid, fonts-wqy-zenhei, fonts-wqy-microhei, fonts-arphic-gbsn00lp, fonts-arphic-gkai00mp, fonts-arphic-ukai, fonts-arphic-uming latex-cjk-all
** Language **
* gcc,g++,gdb,make,autoconf, gfortran, scons
* git, subversion
* openjdk-7-jdk
* python3, pip, virtualenv,pep8,pyflakes
* scala
* haskell-platform
* ruby-full
* clisp, guile
* golang
* nodejs, npm, node-legacy
* r-base, r-recommended, r-cran-\*
* texlive-full
* pandoc,xsltproc,markdown
** Tools **
* scipy, numpy, matpotlib, pandas,sympy, plotly, statsmodels, scikit-learn, nltk(with data), tensorflow
* jupyter notebook
* spark
* mpich2
* storm
* elasticsearch
* kafka/zmq
* couchdb/mongodb/memcached/redis-server

View File

@ -0,0 +1,58 @@
## MPI Example ##
Docklet provides [MPICH2](http://www.mpich.org) to develop MPI program.
Assume the user's vcluster has two nodes: `host-0` and `host-1`.
Open WEB Terminal to check `/etc/hosts` file
```
$ cat /etc/hosts
127.0.0.1 localhost
172.16.0.46 host-0 host-0.tee
172.16.0.45 host-1 host-1.tee
```
We can get the node list by the following command:
```
$ cat /etc/hosts | grep -v localhost | awk '{print $2}'
host-0
host-1
```
Now the WEB Terminal is running on `host-0`, the first node in the
vcluster. Edit `mpihello.c` on `host-0`
```c
#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>
#define MASTER 0
int main (int argc, char *argv[])
{
int numtasks, taskid, len;
char hostname[MPI_MAX_PROCESSOR_NAME];
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
MPI_Comm_rank(MPI_COMM_WORLD,&taskid);
MPI_Get_processor_name(hostname, &len);
printf ("Hello from task %d on %s!\n", taskid, hostname);
if (taskid == MASTER)
printf("MASTER: Number of MPI tasks is: %d\n",numtasks);
MPI_Finalize();
return 0 ;
}
```
Open WEB Terminal and run:
```
$ mpicc mpihello.c -o mpihello
$ mpirun -n 2 -hosts host-0,host-1 mpihello
Hello from task 0 on host-0!
MASTER: Number of MPI tasks is: 2
Hello from task 1 on host-1!
```

View File

@ -0,0 +1 @@
## Spark Example ##

View File

@ -0,0 +1,20 @@
# Workspace #
Docklet Workspace is provided by the integrated open source component of
[Jupyter Notebook](https://github.com/jupyter/notebook).
There are many documents about Jupyter Notebook. Users are recommended
to visit the latest official document
[The Jupyter notebook](http://jupyter-notebook.readthedocs.org/en/latest/index.html) for help, where the [UI Components](http://jupyter-notebook.readthedocs.org/en/latest/ui_components.html) describes Jupyter Notebook user interface.
The Jupyter Notebook dashboard is the entrance of all operations. A
demo snapshot is depicted as follows:
<img src="../images/jupyter-notebook-dashboard.png" width="700"
alt="jupyter notebook dashboard">
This section mainly focuses on the following Workspace functions:
* [Data Management](data.md)
* [File Editing](edit.md)
* [WEB Terminal](terminal.md)
* [Interactive Programming](notebook.md)

View File

@ -0,0 +1,12 @@
## Data Management ##
When users entering their Workspace of Jupyter Notebook, they will see
the tree of filesa and directories. Users can perform **Rename** and
**Delete** operating while selecting some files.
Click **New** - **Folder** can create a folder. Click **New** - ** Text File
** can create a text file with name *untitled.txt*and enter the [File Editing](edit.md) webpage.
Click **Upload**, users can upload their data files to the Workspace.
**Note**: The `~/nfs` directory is the [Dataspace](../concepts/nfs.md). The files in Dataspace will be shared among all containers in the vcluster, and are persistent. Files in other directory are all transient, will be destroyed when the container is deleted.

View File

@ -0,0 +1,15 @@
## File Editing ##
In the Jupyter Notebook dashboard, click text files such as .c .py .md .txt and .log files, will enter the file editing webpage.
Users can rename file, e.g., change *untitled.txt* to *hello.c*.
Those familiar with vi or emacs can choose to use **Vim** or **emacs**
mode in the **Edit** menu. The default edit mode is **Sublime Text**.
The text editor supports tens of programming languages, can
automatically hilight key words, auto indent, etc.
**Note**: The Jupyter Notebook editor now only supports editing small
files. For large files, the click of edition may cause the whole system
** <font color="red">Hangup</font> **

View File

@ -0,0 +1,22 @@
## Interactive Programming ##
Jupyter Notebook has a very attractive feature of **interactive
programming** in browser. It provides a web-based application suitable
for capturing the whole computation process: developing, documenting,
and executing code, as well as communicating the results.
Main features including but not limited to:
* The ability to execute code from the browser, with the results of
computations attached to the code which generated them.
* Displaying the result of computation using rich media representations,
such as HTML, LaTeX, PNG, SVG, etc. For example, publication-quality
figures rendered by the matplotlib library, can be included inline.
* In-browser editing for rich text using the Markdown markup language,
which can provide commentary for the code, is not limited to plain text.
* The ability to easily include mathematical notation within markdown
cells using LaTeX, and rendered natively by MathJax.
Please refer to [The Jupyter Notebook](http://jupyter-notebook.readthedocs.org/en/latest/notebook.html) for more info.
The [Development Demo](../notebook/README.md) section gives some help about programming in
Notebook, mainly programming using Python and R packages.

View File

@ -0,0 +1,48 @@
## WEB Terminal ##
To operate in terminal is necessary for many developers. Almost all
system administrator, programming, debugging and analyzing jobs can be
performed in terminal environment.
In the Jupyter Notebook dashboard, click **New** - **Terminal** will create a new WEB Terminal and then enter the terminal, where users can do their stuffs like editing a file using vi, running a program, etc.
The Jupyter Notebook WEB Terminal has one important feature: running in
the background, even users has closed the WEB Terminal webpage. The
user can find all their live Terminals by clicking **Running** -
**Terminals** in the dashboard. They can re-enter the Terminal by
clicking the name, to recover their work. This feature is very important
for long running jobs.
If the user re-enter the Terminal after a long period of idle time, the Terminal
may show no response to user input. Usually refresh the page will get it
back.
If the user will perform multiple tasks, they can open several
Terminals, or using [tmux](https://tmux.github.io) in one Terminal.
**Note**:
Sometimes, in Chrome, the fonts of terminal may be not monospaced.
This may cause the width of terminal not fixed and commands' output not aligned.
Here is the way to fix it : Chrome -> Settings -> Show Advanced Settings ->
Custom Fonts, choose a monospaced font for **Monospaced Font**
### Install Software ###
Users can install software packages not in the base image. The Docklet
container is based on Ubuntu. The command `apt-get` is used for package
installation.
Example:
```
$ apt-get install clang
```
Users are encouraged to clean the `apt-get` cache to save disk space
```
$ apt-get clean
```
About `apt-get`, please refer to
the [official help](https://help.ubuntu.com/community/AptGet/Howto).

10
userguide/zh/README.md Normal file
View File

@ -0,0 +1,10 @@
Docklet 用户手册
===============
v0.2
2016年2月20日
&copy; 北京大学软工所
http://www.unias.org/trac/docklet

36
userguide/zh/SUMMARY.md Normal file
View File

@ -0,0 +1,36 @@
# Summary
* [Docklet 简介](intro/README.md)
* [什么是 Docklet](intro/what.md)
* [为什么要用 Docklet](intro/why.md)
* [如何使用 Docklet](intro/how.md)
* [基本概念](concepts/README.md)
* [工作区](concepts/workspace.md)
* [虚拟集群](concepts/vcluster.md)
* [镜像](concepts/image.md)
* [服务](concepts/service.md)
* [数据区](concepts/nfs.md)
* [门户系统](portal/README.md)
* [面板](portal/dashboard.md)
* [配置](portal/config.md)
* [状态](portal/status.md)
* [主机](portal/hosts.md)
* [用户](portal/users.md)
* [管理](portal/admin.md)
* [工作区](workspace/README.md)
* [数据管理](workspace/data.md)
* [文件编辑](workspace/edit.md)
* [终端操作](workspace/terminal.md)
* [交互编程](workspace/notebook.md)
* [在终端中开发](software/README.md)
* [安装的软件包列表](software/list.md)
* [C/C++](software/c.md)
* [Java](software/java.md)
* [MPI](software/mpi.md)
* [Spark](software/spark.md)
* [LaTeX](software/latex.md)
* [科学计算与数据处理](notebook/README.md)
* [Python 软件包](notebook/python.md)
* [R 软件包](notebook/r.md)
* [使用Notebook](notebook/nb.md)
* [各种样例](notebook/gallery.md)

View File

@ -0,0 +1,8 @@
# 基本概念 #
Docklet包括如下重要基本概念:
* [工作区](workspace.md)
* [虚拟集群](vcluster.md)
* [镜像](image.md)
* [服务](service.md)
* [数据区](nfs.md)

View File

@ -0,0 +1,19 @@
## 镜像 ##
Docklet的镜像(image)和[Docker](https://github.com/docker/docker) 镜像非常类似,
是一个只读的模板,用来创建容器。但两者又稍有不同。
Docker由于主要关注单机虚拟化其镜像非常自由用户几乎可以任意定制自己的
安装镜像。通常某个Docker镜像只包含某个特定应用框架。
例如一个Docker镜像可以包含一个完整的 ubuntu 操作系统环境,里面仅安装了
Apache 或用户需要的其它应用程序。
Docklet由于关注**工作区**和**虚拟集群**,因此对镜像有特定要求,
不支持用户自由定制上传镜像。Docklet管理员会根据所在单位需求
定制一个或多个公共基础镜像,
里面包含了支持工作区和集群工作必须的软件及配置,以及若干开发语言和计算框架,
如MPI、Spark、Python、R等。由于Docklet公共基础镜像是多用户共享的
因此通常比Docker镜像要大很多。
如果公共基础镜像里没有用户需要的软件,用户可以在容器中安装配置好软件后,
保存自己的工作区,生成自己的私有镜像,并可以发布共享。

View File

@ -0,0 +1,13 @@
## 数据区 ##
在Docklet中每个用户拥有一个持久存储的数据区(Dataspace)
用于存放自己的数据文件。数据区是隔离的,每个用户只能访问自己的数据区。
数据区在虚拟集群的各个容器节点之间是共享的,位于容器节点的 `~/nfs` 目录下。
用户保存自己的工作区为一个镜像的时候,会排除数据区。
数据区由分布式文件系统支撑。除了数据区,容器中保存的其他数据都会随着容器
的销毁而消失,但数据区中的数据会持久存储下去。
建议用户通过工作区,自己及时下载和备份重要的数据。

View File

@ -0,0 +1,9 @@
## 服务 ##
Docklet镜像中的很多软件工具可以直接通过脚本运行
也有一些软件框架以服务(Service)的形式运行,
如[Spark](http://spark.apache.org)、MPI 等。
在用户创建并启动一个容器的时候,这些框架缺省是不启动的,以降低资源消耗。
用户可以对Docklet虚拟集群进行配置选择要启动的服务。
Docklet也支持用户自定义启动服务。

View File

@ -0,0 +1,20 @@
## 虚拟集群 ##
虚拟集群(vcluster: virtual cluster)是Docklet的基础概念
支撑工作区(Workspace)的运行。虚拟集群由若干 LXC 容器组成,
每个容器可被视为一台轻量的Linux主机。通常一个集群中的容器是同构的
即通过同一个镜像生成的,这可以简化集群的管理。
一个虚拟集群能拥有的节点数上限、每个节点的CPU和内存配额取决于用户的级别。
常见的缺省配置为只有1个初始节点。
Docklet虚拟集群的各个节点被设计于尽可能分布在不同的物理主机上
以尽量利用物理资源。这些分布节点彼此可以通信。
一个虚拟集群的所有节点共享一个虚拟子网,通常第一个节点是该子网的网关。
在大多数情况下,用户不需要关注虚拟集群的细节,只需要关注工作区(Workspace)。
在有些场景下,用户需要对虚拟集群进行配置,以支持工作区更好运行。
例如,为了让工作区的运行的代码可以并行执行,需要扩充缺省虚拟集群,
增加新节点。Docklet虚拟集群具有弹性伸缩能力
支持随时为虚拟集群增加或者删除容器节点。

View File

@ -0,0 +1,18 @@
## 工作区 ##
工作区(Workspace)是Docklet的核心概念它是一个基于WEB的交互式工作环境。
Docklet工作区是通过[Jupyter Notebook](https://jupyter.org)实现的,
用户能够创建并分享包含可即刻运行的代码、公式、视频和解释性文本的文档,
可进行数据清洗和转换、数字仿真、概率统计、机器学习、数据分析等工作。
对大多数用户而言,他们不必关心底层的机器、集群、容器、虚拟集群的细节,
他们只需要打开浏览器,进入工作区,就可以交互式地完成大部分的工作,
包括结果的可视化。目前WEB工作区支持40多种编程语言。
下图是[Jupyter Notebook](https://jupyter.org)工作区的一个截图:
<img src="../images/jupyterpreview.png" width="600" alt="jupyter workspace">
对于高级用户工作区还提供了WEB Terminal可进行非可视化的所有其他工作。

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 322 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

View File

@ -0,0 +1,14 @@
# Docklet 简介 #
欢迎来到Docklet的世界!
什么是Docklet
它和大名鼎鼎的Docker有什么关系
用它会带来什么好处?
如何使用Docklet
本章将首先为大家解答上述疑惑。

11
userguide/zh/intro/how.md Normal file
View File

@ -0,0 +1,11 @@
## 如何使用Docklet ##
对于Docklet用户而言只需一个现代浏览器如FirefoxChrome或Safari
即可访问和使用。
Docklet管理员会配置用户登录方式。例如部署在北京大学计算中心的
http://iwork.internetware.org 支持北大帐号登录和内部帐号登录两种方式。
Docklet也支持 Guest 帐号。用户可以用Guest账户体验“工作环境”。
Guest的工作环境会被定期初始化。

View File

@ -0,0 +1,49 @@
## 什么是 Docklet ##
### 定义与结构 ###
Docklet 是北京大学软件工程研究所的一个开源项目,遵从 [新BSD 协议](http://directory.fsf.org/wiki/License:BSD_3Clause),项目代码在 GitHub
上进行维护。
Docklet的目标是实现一种**云化的个人工作区**解决方案,
使企业可以轻松地虚拟化其数据中心,为用户创建个人的虚拟集群,
进而为用户提供一个在**云端**的可定制工作区(Workspace)。
用户只需要一个浏览器,即可随时随地访问企业内部自己的工作区,
在线进行代码编写、调试运行、数据管理、数据分析、结果可视化等工作。
Docklet的核心是基于Linux容器LXC技术和软件定义网络技术的容器集群。
它在LXC的基础上进行了进一步的封装让用户不仅不需要关心单个容器的管理细节
更无须关注各个分布容器之间的网络通信。
这样一个容器集群提供了一种物理集群虚拟化方案,
支持安装运行绝大多数已有的单机或分布式Linux应用形成个人的开发工作区。
用户通过Docklet集成的[Jupyper Notebook](https://github.com/jupyter/notebook)
访问自己的工作区,通过浏览器完成开发运行等工作。
Docklet的结构如下图所示:
<img src="../images/docklet-architecture.jpg" width="500"
alt="docklet-architecture">
<!--
![docklet architecture](../images/docklet-architecture.jpg)
-->
### Docklet与Docker ###
Docklet和[Docker](https://github.com/docker/docker) 没有直接关系,
它们是两个完全不同的项目,设计目标也完全不同。
Docker的目标是实现轻量级的操作系统虚拟化解决方案
位于云计算的[IaaS层](https://en.wikipedia.org/wiki/Cloud_computing)。
Docker在LXC的基础上进行了进一步的封装
使得[用户操作 Docker 的容器就像操作一个快速轻量级的虚拟机一样简单
](https://www.gitbook.com/book/yeasy/docker_practice)。
虽然其后增加了网络集群能力,但本质上仍然是面向单机系统的,
位于软件栈的操作系统层。
Docklet的目标是实现一种开发者工作区的云化解决方案
涵盖[SaaS、PaaS和IaaS层](https://en.wikipedia.org/wiki/Cloud_computing)。
Docklet的基础是LXC容器集群而不是Docker。
对Docklet用户而言他们直接通过浏览器进行软件开发、调试
测试工作,面向的是运行于虚拟集群中的各种开发工具,这些工具处于软件栈的应用层。

79
userguide/zh/intro/why.md Normal file
View File

@ -0,0 +1,79 @@
## 为什么要用 Docklet ##
作为一种“**工作区云**"Docklet有很多优点。
首先Docklet 提供的是一个面向开发者的虚拟工作区而不仅是IaaS或PaaS
对用户更友好。其次Docklet面向虚拟集群的工作方式对物理资源的利用效率很高
使用简便,对管理员非常友好。
具体说来Docklet 具有如下的优点和特色:
### 适用广泛 ###
Docklet并非是针对大型数据中心设计的而是适合于微小型的数据中心
即规模在百台物理机以下、由高速网络连接、节点具有较高性能的集群,
这是大多数中小企业和研究机构的常见配置。这使得Docklet拥有广泛的适用性。
### 多个计算框架共享集群资源 ###
传统的MPI集群及最近的Spark集群等通常是专用的
机器难以在不同的计算框架间共享。[Mesos](http://mesos.apache.org)和
[Yarn](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html)分别提供了一种共享集群资源的机制,但它们是通过为计算框架进行适配实现的,
需要为每个要支持的计算框架增加额外的编程工作。Docklet通过将集群虚拟化
提供了一种更灵活易用的共享资源的方式,
各种计算框架无需修改就可以无缝运行在虚拟集群中,非常便利灵活。
### 动态弹性配置资源 ###
Docklet的虚拟集群支持弹性配置资源。用户只需运行一个命令或者进行一个点击
就可以为其虚拟集群**瞬间**增加一个和现有容器功能相同的容器节点。
这对于Spark这样支持弹性资源使用方式的计算框架而言非常有用。
Docklet还能监测用户的活动智能发现非工作用户自动调整资源占用配额
以取得更好的资源利用率和用户使用体验。
### 消耗的网络资源少 ###
Docklet使用了软件定义网络技术内部是一个虚拟的IP网络
外部只需要配置**一个公共IP地址/域名**。用户无需其他软件,
只需要在浏览器中访问该公共IP地址/域名,就可以自动穿透单位防火墙,
访问运行在单位内部的私有虚拟集群。
### 使用简单 ###
Docklet主要是面向单位内部的开发者用户的
通过一个公共的基础镜像,为用户预置了该单位软件开发所需的常用软件资源。
用户可以很容易只需一个浏览器就能进入工作区,
直接就可以进行编程、调试、运行工作,
避免了安装配置物理集群、安装配置软件系统、管理镜像等复杂繁琐的任务,使用简便。
这对于非IT专业的用户而言尤为重要。
### 适合做大数据等实验测试 ###
Docklet使用简便意味着Docklet可以是很理想的实验测试平台。
例如用户可以针对一个较小的数据集Gigabyte级别进行大数据程序的测试调试
,成功后再部署到生产系统运行。对于很多数据量不大的数据分析任务而言,
可以直接在Docklet上运行获得最终结果。
Docklet也是一个理想的学习平台
内置的各种语言工具、计算框架可以帮助用户直接进行各种编程的练习与实验。
### 方便定制与扩展 ###
Docklet的系统管理员可以定制特定于单位需求的公共基础镜像
用户也可以自行安装配置所需的特定软件,
然后可以将自己的工作区保存为私有镜像并共享发布,支持其他用户使用。
因此,具有良好的定制性与扩展性。
### 支持多用户 ###
Docklet的虚拟集群天生支持多用户每个用户拥有自己独立的私有虚拟集群
每个集群可以安装运行独立的MPI、Spark计算框架而互不干扰
从而可以满足不同用户不同的需求。
### LXC具有的优点 ###
Docklet具有Docker这样基于LXC技术的虚拟化方案所天生具有的诸多优点
如启动快,容器的启动可以在秒级实现;对系统资源的利用率很高,
一台主机上可以同时运行成百上千个容器;应用性能高,消耗的系统资源少;
交付快速,管理简单等。

View File

@ -0,0 +1,34 @@
# 科学计算与数据处理 #
Docklet 主要通过集成的 [Python](https://www.python.org) 和 [R](http://www.r-project.org)
语言软件包支持进行科学计算与数据处理。大部分工作都可以在Jupyter Notebook中交互完成
并将结果以可视化形式在浏览器中显示。
互联网上有很多关于如何使用 Python 和 R 进行大数据处理、科学计算、机器学习、
自然语言处理、统计分析、数据挖掘、图计算、数据可视化等任务的文章,
主要是对各种软件包和工具的介绍,这对于初学用户而言非常有价值。
[CONTINUUM](https://www.continuum.io) 的
[ANACONDA](https://www.continuum.io/why-anaconda)是一个计算分析平台,
分门别类集成了很多开源 Python 软件包,以及部分 R 软件包。
[ENTHOUGHT](https://www.enthought.com) 的
[Canopy](https://www.enthought.com/products/canopy/)是一个和CONTINUUM类似的系统
集成了很多开源的 Python软件包。
[RStudio](https://www.rstudio.com) 有 [RStudio
IDE](https://www.rstudio.com/products/rstudio2/) 和
[R 软件包](https://www.rstudio.com/products/rpackages/) 的介绍。
很多和可视化、IDE有关的工具需要在本地操作系统安装运行。Docklet 中的
Python 软件包和 R 软件包大部分可以在 Jupyter Notebook 中运行。用户也可以
通过 `pip3` 等工具自行安装缺失的软件包。
Docklet的Python 和 R 软件包列表如下:
* [Python 软件包](python.md)
* [R 软件包](r.md)
如何在 Jupyter Notebook 中进行科学计算等活动的教程和示例请见 [这里
](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks) 和 [这里](http://nbviewer.jupyter.org) 。

View File

@ -0,0 +1,587 @@
## 一些有序的 Notebooks 示例 ##
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks 收集了一些用 Notebook 进行各种工作的样例。Docklet 预安装了常用的 [Python3](python.md) 和
[R](r.md) 软件包,如果某些软件包没有安装,请用户通过 `pip3` 等命令自行安装。
1. [Entire books or other large collections of notebooks on a topic](#entire-books-or-other-large-collections-of-notebooks-on-a-topic)
* [Introductory Tutorials](#introductory-tutorials)
* [Programming and Computer Science](#programming-and-computer-science)
* [Statistics, Machine Learning and Data Science](#statistics-machine-learning-and-data-science)
* [Mathematics, Physics, Chemistry, Biology](#mathematics-physics-chemistry-biology)
* [Earth Science and Geo-Spatial data](#earth-science-and-geo-spatial-data)
* [Linguistics and Text Mining](#linguistics-and-text-mining)
* [Signal Processing](#signal-processing)
* [Engineering Education](#engineering-education)
1. [Scientific computing and data analysis with the SciPy Stack](#scientific-computing-and-data-analysis-with-the-scipy-stack)
* [General topics in scientific computing](#general-topics-in-scientific-computing)
* [Social data](#social-data)
* [Psychology and Neuroscience](#psychology-and-neuroscience)
* [Machine Learning, Statistics and Probability](#machine-learning-statistics-and-probability)
* [Physics, Chemistry and Biology](#physics-chemistry-and-biology)
* [Economics and Finance](#economics-and-finance)
* [Earth science and geo-spatial data](#earth-science-and-geo-spatial-data)
* [Data visualization and plotting](#data-visualization-and-plotting)
* [Mathematics](#mathematics)
* [Signal and Sound Processing](#signal-and-sound-processing)
* [Natural Language Processing](#natural-language-processing)
* [Pandas for data analysis](#pandas-for-data-analysis)
1. [General Python Programming](#general-python-programming)
1. [Notebooks in languages other than Python](#notebooks-in-languages-other-than-python)
* [Julia](#julia)
* [Haskell](#haskell)
* [Ruby](#ruby)
* [Perl](#perl)
1. [Miscellaneous topics about doing various things *with* the Notebook itself](#miscellaneous-topics-about-doing-various-things-with-the-notebook-itself)
1. [Reproducible academic publications](#reproducible-academic-publications)
1. [Other publications using the Notebook](#other-publications-and-conference-abstracts-that-explicitly-use-the-notebook)
1. [Data-driven journalism](#data-driven-journalism)
1. [Whimsical notebooks](#whimsical-notebooks)
1. [Videos of IPython being used in the wild](#videos-of-ipython-being-used-in-the-wild)
## Entire books or other large collections of notebooks on a topic
### Introductory Tutorials
* First things first, how to [run code in the notebook](https://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Running%20Code.ipynb). There is also a general [collection of notebooks](https://nbviewer.jupyter.org/github/ipython/ipython/blob/master/examples/IPython%20Kernel/Index.ipynb) from IPython. Another useful one from this collection is an explanation of our [rich display system](https://nbviewer.jupyter.org/github/ipython/ipython/blob/master/examples/IPython%20Kernel/Custom%20Display%20Logic.ipynb).
* A [great matplotlib tutorial](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-4-Matplotlib.ipynb), part of the fantastic [Lectures on Scientific Computing with Python](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/tree/master) by [J.R. Johansson](https://github.com/jrjohansson).
* The code of the [IPython mini-book](https://github.com/rossant/ipython-minibook) by C. Rossant, introducing IPython, NumPy, SciPy, Pandas and matplotlib for interactive computing and data visualization.
* [Python Tutorial](https://github.com/rajathkumarmp/Python-Lectures) by [Rajath Kumar M P](https://github.com/rajathkumarmp)
### Programming and Computer Science
* [Introduction to Programming (using Python)](http://nbviewer.ipython.org/github/ehmatthes/intro_programming/blob/master/notebooks/index.ipynb), an entire introductory Python course written by [Eric Matthes](http://peak5390.wordpress.com/about). [This post](http://peak5390.wordpress.com/2013/09/22/how-ipython-notebook-and-github-have-changed-the-way-i-teach-python/) explains the educational context in an Alaskan high school where Eric is a teacher.
* [Python for Developers](http://ricardoduarte.github.io/python-for-developers), a complete book on Python programming by [Ricardo Duarte](https://github.com/ricardoduarte). Note the book also exists [in Portuguese](http://ricardoduarte.github.io/python-para-desenvolvedores).
* [CS1001.py - Extended Introduction to Computer Science](https://github.com/yoavram/CS1001.py). Recitations from Tel-Aviv University introductory course to computer science, assembled as IPython notebooks by [Yoav Ram](http://www.yoavram.com/).
* [Exploratory Computing with Python](http://mbakker7.github.io/exploratory_computing_with_python/), a set of 15 Notebooks that cover exploratory computing, data analysis, and visualization. No prior programming knowledge required. Each Notebook includes a number of exercises (with answers) that should take less than 4 hours to complete. Developed by Mark Bakker for undergraduate engineering students at the Delft University of Technology.
* [Understanding evolutionary strategies and covariance matrix adaptation](http://nbviewer.ipython.org/github/lmarti/evolutionary-computation-course/blob/master/AEC.04%20-%20Evolutionary%20Strategies%20and%20Covariance%20Matrix%20Adaptation.ipynb), from the [Advanced Evolutionary Computation: Theory and Practice](http://nbviewer.ipython.org/github/lmarti/evolutionary-computation-course/tree/master/) course by [Luis Martí](http://lmarti.com).
### Statistics, Machine Learning and Data Science
* [AM207: Monte Carlo Methods, Stochastic Optimization](http://am207.org): a complete course by Verena Kaynig-Fittkau and Pavlos Protopapas from Harvard, with all lecture materials and homework sets as notebooks.
* [An introduction to Bayesian inference](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1.ipynb), this is just chapter 1 in an ongoing book titled [Probabilistic Programming and Bayesian Methods for Hackers Using Python and PyMC](http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers), by [Cameron Davidson-Pilon](http://camdp.com/).
* [Learn Data Science](http://learnds.com), an entire self-directed course by [Nitin Borwankar](https://github.com/nborwankar).
* [IPython Cookbook](http://ipython-books.github.io/cookbook/) by [Cyrille Rossant](http://cyrille.rossant.net/), a comprehensive guide to Python for Data Science. The code of the 100 recipes is available on [the GitHub repository](https://github.com/ipython-books/cookbook-code).
* [An introduction to machine learning with Python and scikit-learn](http://nbviewer.ipython.org/github/temporaer/tutorial_ml_gkbionics/blob/master/2%20-%20KMeans.ipynb) ([repo and overview](https://github.com/amueller/tutorial_ml_gkbionics)) by [Hannes Schulz](https://github.com/temporaer) and [Andreas Mueller](https://github.com/amueller).
* [Clustering and Regression](http://nbviewer.ipython.org/github/amplab/datascience-sp14/blob/master/hw2/HW2.ipynb), part of the UC Berkeley 2014 [Introduction to Data Science](http://amplab.github.io/datascience-sp14/) course taught by [Michael Franklin](http://www.cs.berkeley.edu/~franklin/).
* [Neural Networks](http://nbviewer.ipython.org/github/masinoa/machine_learning/blob/master/04_Neural_Networks.ipynb), part of a collection on [machine learning](https://github.com/masinoa/machine_learning) by [Aaron Masino](https://github.com/masinoa).
* [An introduction to Pandas](http://nbviewer.ipython.org/urls/bitbucket.org/hrojas/learn-pandas/raw/master/lessons/01%20-%20Lesson.ipynb), part of an [11-lesson tutorial on Pandas](https://bitbucket.org/hrojas/learn-pandas), by [Hernán Rojas](https://bitbucket.org/hrojas).
* The [Statsmodels Project](http://statsmodels.sourceforge.net) has two excellent collections of examples: [in their official documentation](http://statsmodels.sourceforge.net/devel/examples/index.html) and [extra ones in their wiki](https://github.com/statsmodels/statsmodels/wiki/Examples#user-contributed-examples). Too many there to directly duplicate here, but they provide great learning materials on statistical modeling with Python.
* [Machine Learning with the Shogun Toolbox](http://cloud.shogun-toolbox.org). This is a complete *service* that includes a ready-to-run IPython instance with a collection of notebooks illustrating the use of the [Shogun Toolbox](http://shogun-toolbox.org). Just log in and start running the examples.
* [Python for Data Analysis](https://github.com/ResearchComputing/Meetup-Fall-2013), an introductory collection from the [CU Boulder Research Computing Group](http://researchcomputing.github.io/).
* [The Kaggle bulldozers competition example](http://danielfrg.github.io/blog/2013/03/07/kaggle-bulldozers-basic-cleaning), one of a set on tutorials on exploratory data analysis with the [copper toolkit](https://github.com/danielfrg/copper#copper) by [Daniel Rodríguez](http://danielfrg.github.io)/
* [Understanding model reliability](http://nbviewer.ipython.org/github/mwaskom/Psych216/blob/master/week6_tutorial.ipynb), part of a complete [course on statistics and data analysis for psychologists](https://github.com/mwaskom/Psych216) by [Michael Waskom](https://github.com/mwaskom).
* [Graphical Representations of Linear Models](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/linear_models.ipynb), an illustration of the [Seaborn statistical visualization library](https://github.com/mwaskom/seaborn), that also includes [Visualizing distributions of data](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/plotting_distributions.ipynb) and [Representing variability in timeseries plots](http://nbviewer.ipython.org/github/mwaskom/seaborn/blob/master/examples/timeseries_plots.ipynb). By [Michael Waskom](https://github.com/mwaskom).
* [Desperately Seeking Silver](http://nbviewer.ipython.org/github/cs109/content/blob/master/HW2.ipynb), one of the homework sets for Harvard's [CS 109 Data Science course](http://cs109.org).
* [IPython Notebooks for 'An Introduction to Statistical Learning with Applications in R'](https://github.com/JWarmenhoven/ISLR-python), Python code for a selection of tables, figures and LAB sections from the book by James, Witten, Hastie, Tibshirani (2013).
* [Python Notebooks for StatLearning Exercises](https://github.com/sujitpal/statlearning-notebooks), Python implementations of the R labs for the [StatLearning: Statistical Learning](https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/about) online course from Stanford University taught by Profs Trevor Hastie and Rob Tibshirani.
* [Applied Predictive Modeling with Python](http://nbviewer.ipython.org/github/leig/Applied-Predictive-Modeling-with-Python/tree/master/notebooks/), Python implementations of the examples (originally written in R) from a famous introductory book, [Applied Predictive Modeling](http://appliedpredictivemodeling.com), by Max Kuhn and Kjell Johnson.
* A collection of [four courses in foundations of data science, algorithms and databases](http://nbviewer.ipython.org/github/ledeprogram/courses/tree/master) from multiple faculty at [Columbia University's Lede Program](http://www.journalism.columbia.edu/page/1058-the-lede-program-an-introduction-to-data-practices/906).
* [SciPy and OpenCV as an interactive computing environment for computer vision](http://nbviewer.ipython.org/github/thsant/scipy4cv) by [Thiago Santos](http://ttsantos.net), a tutorial presented at [SIBGRAPI 2014](http://emap.fgv.br/sibgrapi-2014/tutorials.html).
* [Kalman and Bayesian Filters in Python](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb), by [Roger Labbe](https://github.com/rlabbe).
* [Machine learning in Python](http://nbviewer.ipython.org/github/jdwittenauer/ipython-notebooks/blob/master/ML-Exercise1.ipynb), a series based on Andrew Ng's Coursera class on machine learning. Part of a [larger collection of data science notebooks](https://github.com/jdwittenauer/ipython-notebooks) by [John Wittenauer](https://github.com/jdwittenauer).
* [An example machine learning notebook](http://nbviewer.jupyter.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/example-data-science-notebook/Example%20Machine%20Learning%20Notebook.ipynb), by [Randal. S. Olson](http://www.randalolson.com/), part of a [collection in Data Analysis and Machine Learning](http://nbviewer.jupyter.org/github/rhiever/Data-Analysis-and-Machine-Learning-Projects).
### Mathematics, Physics, Chemistry, Biology
* A [single-atom laser model](http://nbviewer.ipython.org/github/jrjohansson/qutip-lectures/blob/master/Lecture-2B-Single-Atom-Lasing.ipynb). This is one of a complete set of [lectures on quantum mechanics and quantum optics using QuTiP](http://nbviewer.ipython.org/github/jrjohansson/qutip-lectures) by [J.R. Johansson](https://github.com/jrjohansson).
* [2-d rigid-body transformations](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/Transformation2D.ipynb). This is part of [Scientific Computing in Biomechanics and Motor Control](https://github.com/demotu/BMC), a complete collection of notebooks by [Marcos Duarte](https://github.com/demotu).
* Astrophysical simulations and analysis with [yt](http://yt-project.org): a collection of example notebooks on using various codes that yt interfaces with: [Enzo](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/ac275ee8a462425c93b36e330e243705.ipynb), [Gadget](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9ab12c55422bb3afdc.ipynb), [RAMSES](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb), [PKDGrav](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb) and [Gasoline](http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/9fe5b4bc68ce48d5aa665edbc4ec3aa3.ipynb). *Note:* the yt site currently throws an SSL warning, they seem to have an outdated or self-signed certificate.
* [Working with Reactions](http://nbviewer.ipython.org/gist/4316430), part of a set of tutorials on [cheminformatics and machine learning with the rdkit project](http://code.google.com/p/rdkit/wiki/UGM2012Tutorials), by Greg Landrum.
* [CFD Python: 12 steps to Navier-Stokes](http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes). A complete set of lectures on Computational Fluid Dynamics, from 1-d linear waves to full 2-d Navier-Stokes, by [Lorena Barba](http://lorenabarba.com).
* [AeroPython: Aerodynamics-Hydrodynamics with Python](https://github.com/barbagroup/AeroPython), a complete course taught at George Washington University by [Lorena Barba](http://lorenabarba.com).
* [Practical Numerical Methods with Python](https://github.com/numerical-mooc/numerical-mooc), a collection of learning modules (each consisting of several IPython Notebooks) for a course in numerical differential equations taught at George Washington University by [Lorena Barba](http://lorenabarba.com). Also offered as a "massive, open online course" (MOOC) on the [GW SEAS Open edX](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about) platform.
* [pyuvvis: tools for explorative spectroscopy](https://github.com/hugadams/pyuvvis), spectroscopy library built for integration ipython notebooks, matplotlib and pandas.
* [HyperPython: a practical introduction to the solution of hyperbolic conservation laws](http://nbviewer.ipython.org/github/ketch/HyperPython/tree/master/), a course by [David Ketcheson](http://davidketcheson.info).
* [An Introduction to Applied Bioinformatics](http://caporasolab.us/An-Introduction-To-Applied-Bioinformatics): Interactive lessons in bioinformatics, by [Greg Caporaso](http://caporasolab.us).
* Colour science computations with [colour](https://github.com/colour-science/colour), a Python package implementing a comprehensive number of colour theory transformations and algorithms supported by a [dedicated collection of IPython Notebooks](http://nbviewer.ipython.org/github/colour-science/colour-ipython/blob/master/notebooks/colour.ipynb). More colour science related [IPython Notebooks](http://nbviewer.ipython.org/github/colour-science/colour-website/tree/master/ipython/) are available on [colour-science.org](http://colour-science.org/).
* The [notebooks](https://github.com/tiagoantao/bioinf-python/blob/master/notebooks/Welcome.ipynb) from the Book [Bioinformatics with Python Cookbook](http://www.amazon.com/Bioinformatics-Python-Cookbook-Tiago-Antao/dp/1782175113), covering several fields like Next-Generation Sequencing, Population Genetics, Phylogenetics, Genomics, Proteomics and Geo-referenced information.
### Earth Science and Geo-Spatial data
* [EarthPy](http://earthpy.org), a collection of IPython notebooks with a focus on Earth Sciences, from [whale tracks](http://earthpy.org/analyzing-whale-tracks.html) to the [flow of the Amazon](http://earthpy.org/earthpy-basemap-amazon.html).
* [Python for Geosciences](https://github.com/koldunovn/python_for_geosciences), a tutorial series aimed at the Earth Sciences community, by [Nikolay Koldunov](http://earthpy.org/author/nikolay-koldunov.html).
* [Find graffiti close to NY subway entrances](http://nbviewer.ipython.org/github/invisibleroads/crosscompute-tutorials/blob/master/computational-analysis/300%20Count%20graffiti%20sightings%20within%20100%20feet%20of%20a%20subway%20entrance.ipynb), one of a rich [collection of notebooks on large-scale data analysis](https://github.com/invisibleroads/crosscompute-tutorials), by [Roy Hyunjin Han](https://github.com/invisibleroads).
* [Logistic models of well switching in Bangladesh](http://nbviewer.ipython.org/github/carljv/Will_it_Python/blob/master/ARM/ch5/arsenic_wells_switching.ipynb), part of the ["Will it Python"](http://slendrmeans.wordpress.com/will-it-python) blog series ([repo](https://github.com/carljv/Will_it_Python)) on Machine Learning and data analysis in Python. By Carl Vogel.
* [Estimated likelihood of observing a large earthquake on a continental lowangle normal fault and implications for lowangle normal fault activity](http://nbviewer.ipython.org/github/cossatot/lanf_earthquake_likelihood/blob/master/notebooks/lanf_manuscript_notebook.ipynb), an executable version of a paper by Richard Styron and Eric Hetland published in *Geophysical Research Letters*, on earthquake probabilities.
* [python4oceanographers](https://ocefpaf.github.io/python4oceanographers/), a blog demonstrating analyses in physical oceanography from [resource-demanding numerical computations with functions in compiled languages](https://ocefpaf.github.io/python4oceanographers/blog/2015/10/05/isosurfaces/) to specialized [tidal analysis](https://ocefpaf.github.io/python4oceanographers/blog/2015/02/16/utide/) to [visualization of various geo data using fancy things like interactive maps](https://ocefpaf.github.io/python4oceanographers/blog/2015/07/13/interactive_geo/).
### Linguistics and Text Mining
* [Workshop on text analysis](http://nbviewer.ipython.org/github/nealcaren/workshop_2014/tree/master/notebooks/) by [Neal Caren](http://nealcaren.web.unc.edu).
* [Detecting Algorithmically Generated Domains](http://nbviewer.ipython.org/github/ClickSecurity/data_hacking/blob/master/dga_detection/DGA_Domain_Detection.ipynb), part of the [Data Hacking](http://clicksecurity.github.io/data_hacking) collection on security-oriented data analysis with IPython & friends.
* [Mining the Social Web (2nd Edition)](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition). A complete collection of notebooks accompanying [Matthew Russel's book](http://shop.oreilly.com/product/0636920030195.do) by O'Reilly.
### Signal Processing
* [Sound Analysis with the Fourier Transform](https://github.com/calebmadrigal/FourierTalkOSCON). A set of IPython Notebooks by [Caleb Madrigal](http://calebmadrigal.com) to explain what the Fourier Transform is and how to use it for basic audio processing applications.
* [An introduction to Compressed Sensing](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing/blob/master/Compressive_Sampling.ipynb), part of [Python for Signal Processing](http://nbviewer.ipython.org/github/unpingco/Python-for-Signal-Processing): an entire book (and [blog](http://python-for-signal-processing.blogspot.com)) on the subject by Jose Unpingco.
* [Kalman and Bayesian Filters in Python](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb). A textbook and accompanying filtering library on the topic of Kalman filtering and other related Bayesian filtering techniques.
* [Classify human movements using Dynamic Time Warping & K Nearest Neighbors:](http://nbviewer.ipython.org/github/markdregan/K-Nearest-Neighbors-with-Dynamic-Time-Warping/blob/master/K_Nearest_Neighbor_Dynamic_Time_Warping.ipynb) Signals from a smart phone gyroscope and accelerometer are used to classify if the person is running, walking, sitting standing etc. This IPython notebook contains a python implementation of DTW and KNN algorithms along with explanations and a practical application.
* [Digital Signal Processing](https://github.com/spatialaudio/digital-signal-processing-lecture) A collection of notebooks that accompanies a masters course on the topic.
### Engineering Education
* [Introduction to Chemical Engineering Analysis](http://jckantor.github.io/CBE20255/) by [Jeff Kantor](http://jckantor.github.io/). A collection of IPython notebooks illustrating topics in introductory chemical engineering analysis, including stoichiometry, generation-consumption analysis, mass and energy balances.
## Scientific computing and data analysis with the SciPy Stack
### General topics in scientific computing
* [Algorithms in IPython notebooks](https://github.com/rasbt/algorithms_in_ipython_notebooks), by [Sebastian Raschka](https://github.com/rasbt)
* [Comparing the performance of Python compilers - Cython vs. Numba vs. Parakeet](_nbs/day4_2_cython_numba_parakeet.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt)
* [A Crash Course in Python for Scientists](http://nbviewer.ipython.org/gist/5920182), by Sandia's [Rick Muller](http://www.cs.sandia.gov/~rmuller).
* [A gentle introduction to scientific programming in Python, biased towards biologists](http://nbviewer.ipython.org/url/atwallab.cshl.edu/teaching/QBbootcamp3.ipynb), by [Mickey Atwal, Cold Spring Harbor Laboratory](http://atwallab.cshl.edu).
* [Python for Data Science](http://nbviewer.ipython.org/github/gumption/Python_for_Data_Science/blob/master/Python_for_Data_Science_all.ipynb), a self-contained mini-course with exercises, by [
Joe McCarthy](http://interrelativity.com/joe).
* [First few lectures of the UW/Coursera course on Data Analysis](http://nbviewer.ipython.org/github/fonnesbeck/ComputationalMethodsCourse/blob/master/Lecture%201.ipynb). ([Repo](https://github.com/fonnesbeck/ComputationalMethodsCourse)) by Chris Fonnesbeck.
* [CythonGSL: a Cython interface for the GNU Scientific Library (GSL)](http://nbviewer.ipython.org/github/twiecki/CythonGSL/blob/master/examples/cython_gsl_ipythonnb.ipynb) ([Project repo](https://github.com/twiecki/CythonGSL), by Thomas Wiecki.
* [Using Numba to speed up numerical codes](http://nbviewer.ipython.org/gist/3914904). And another Numba example: [self-organizing maps](http://nbviewer.ipython.org/gist/3407544).
* [Numpy performance tricks](http://nbviewer.ipython.org/gist/4645217), and [blog post](http://cyrille.rossant.net/numpy-performance-tricks/), by [Cyrille Rossant](http://cyrille.rossant.net/).
* [IPython Parallel Push/Execute/Pull Demo](http://nbviewer.ipython.org/gist/3866987) by Justin Riley.
* [Understanding the design of the R "formula" objects](http://nbviewer.ipython.org/url/perrin.dynevor.org/exploring_r_formula.ipynb) by Matthew Brett.
* [Comparing different approaches to evolutionary simulations](http://nbviewer.ipython.org/url/www.slideviper.oquanta.info/test/simulation%20frameworks.ipynb). Also available [here](http://slideviewer.herokuapp.com/urls/github.com/yoavram/ipython-notebooks/raw/master/simulation%20frameworks.ipynb) to better visualization. The notebook was converted to a HTML presentation using an old nbconvert with the first developing implementation of `reveal` converter. By Yoav Ram.
* [The Traveling Salesperson Problem](http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipynb) by [Peter Norvig](http://norvig.com).
* [A git tutorial targeted at scientists](http://nbviewer.ipython.org/github/fperez/reprosw/blob/master/Version%20Control.ipynb) by Fernando Perez.
* [Running MATLAB in an IPython Notebook](http://nbviewer.ipython.org/github/plotly/IPython-plotly/blob/master/See%20more/MATLAB_Wrapper_Examples.ipynb), using [pymatbridge](https://pypi.python.org/pypi/pymatbridge).
* [Interactive Curve-Fitting](http://nbviewer.ipython.org/gist/danielballan/1c2db3d4f2f7780cf52f) The `lmfit` package provides a widget-based interface to the curve-fitting algorithms in SciPy.
* [A visual guide to the Python Spark API for distributed computing](http://nbviewer.ipython.org/github/jkthompson/pyspark-pictures/blob/master/pyspark-pictures.ipynb) by Jeff Thompson
### Social data
* [Survival Analysis](http://nbviewer.ipython.org/github/CamDavidsonPilon/lifelines/blob/master/docs/Survival%20Analysis%20intro.ipynb), an illustration of the [lifelines library](https://github.com/CamDavidsonPilon/lifelines), by [Cam Davidson Pilon](https://github.com/CamDavidsonPilon).
* A reconstruction of [Nate Silver's 538 model for the 2012 US Presidential Election](http://nbviewer.ipython.org/github/jseabold/538model/blob/master/silver_model.ipynb), by Skipper Seabold (complete [repo](https://github.com/jseabold/538model)).
* [Data about the Sandy Hook massacre in Newtown, Conneticut](http://nbviewer.ipython.org/url/www.brianckeegan.com/data/Shootings/Shootings.ipynb), which accompanies a more detailed [blog post on the subject](http://www.brianckeegan.com/2012/12/sandy-hook-school-massacre/). Here are the [notebook and accompanying data](http://www.brianckeegan.com/data/Shootings/Shootings.ipynb). By Brian Keegan.
* More on [gun violence analysis with Wikipedia data](http://nbviewer.ipython.org/gist/4358066).
* [An analysis of the Gaza-Israel 2012 crisis](http://nbviewer.ipython.org/gist/4121857).
* [Ranking NFL Teams](http://nbviewer.ipython.org/github/seanjtaylor/NFLRanking/blob/master/NFL%20Rankings.ipynb). The [full repo](https://github.com/seanjtaylor/NFLRanking) also includes an explanatory slideshow. By Sean Taylor.
* [Automated processing of news media and generation of associated imagery](http://nbviewer.ipython.org/url/mhermans.net/files/tmp/demo_rdf_HLN.ipynb).
* [An analysis of national school standardized test data in Colombia using Pandas](http://nbviewer.ipython.org/url/finiterank.com/saber/saber.ipynb) (in Spanish). By [Javier Moreno](http://finiterank.com).
* [Getting started with GDELT](http://nbviewer.ipython.org/github/dmasad/GDELT_Intro/blob/master/Getting_Started_with_GDELT.ipynb), by [David Masad](https://github.com/dmasad). [GDELT](http://eventdata.psu.edu/data.dir/GDELT.html) is a dataset containing more than 200-million geolocated events with global coverage for 1979 to the present. Another GDELT example from David, that nicely [integrates mapping visualizations](http://nbviewer.ipython.org/github/dmasad/GDELT_Intro/blob/master/GDELT_Mapping.ipynb).
* [Titanic passengers, coal mining disasters, and vessel speed changes](http://nbviewer.ipython.org/gist/fonnesbeck/8495259), by [Christopher Fonnesbeck](http://biostat.mc.vanderbilt.edu/wiki/Main/ChrisFonnesbeck)
* A geographic analysis of [Indonesian conflicts in 2012](http://nbviewer.ipython.org/github/herrfz/gdelt/blob/master/indn_connection.ipynb) with GDELT, by [herrfz](https://github.com/herrfz).
* [Bioinformatic Approaches to the Computation of Poetic Meter](http://nbviewer.ipython.org/github/asp49/meter/blob/graph/Shared%20Horizons%20Presentation.ipynb), by [A. Sean Pue](https://github.com/asp49), [C. Titus Brown](https://github.com/ctb) and [Tracy Teal](https://github.com/tracyteal).
* Analyzing the [Vélib dataset from Paris](http://nbviewer.ipython.org/gist/5520933), by [Cyrille Rossant](https://github.com/rossant) (Vélib is Paris' [bicycle-sharing program](http://en.wikipedia.org/wiki/V%C3%A9lib')).
* [Using Python to see how the Times writes about men and women](http://nbviewer.ipython.org/gist/5105037), by [Neal Caren](https://twitter.com/HaphazardSoc).
* [Exploring graph properties of the Twitter stream with twython and NetworkX](http://nbviewer.ipython.org/gist/5681541/TwitterGraphs.ipynb), by [F. Perez](http://fperez.org) (complete [gist repo with utilities here](https://gist.github.com/fperez/5681541).)
* [Kaggle Competition: Titanic Machine Learning from Disaster](http://nbviewer.ipython.org/github/agconti/kaggle-titanic/blob/master/Titanic.ipynb). By [Andrew Conti](https://github.com/agconti).
* [How clean are San Francisco's restaurants?](http://nbviewer.ipython.org/github/Jay-Oh-eN/happy-healthy-hungry/blob/master/h3.ipynb), a data science tutorial that accompanies a [blog post](http://blog.zipfianacademy.com/post/57158627293/how-to-data-science-mapping-sf-restaurant-inspection) from [Zipfian Academy](http://zipfianacademy.com/).
* [NYT gender wage gap and US crime by state](http://nbviewer.ipython.org/github/plotly/IPython-plotly/blob/master/Bubble%20Chart%20Explorer.ipynb).
* [Predicting usage of the subway system in NYC](http://nbviewer.ipython.org/url/www.asimihsan.com/articles/Intro%20to%20Data%20Science%20-%20Final%20Project.ipynb), a [final project](http://blog.udacity.com/2014/05/intro-to-data-science-tools-to-ask.html) for the Udacity Intro to Data Science Course, by [Asim Ihsan](http://www.asimihsan.com).
* [An exploratory statistical analysis of the 2014 World Cup Final](http://nbviewer.ipython.org/github/rjtavares/football-crunching/blob/master/notebooks/an%20exploratory%20data%20analysis%20of%20the%20world%20cup%20final.ipynb), by [Ricardo Tavares](https://github.com/rjtavares). Part of a [notebook collection on football (aka soccer) analysis](http://nbviewer.ipython.org/github/rjtavares/football-crunching/tree/master/).
* [San Francisco's Drug Geography](http://nbviewer.ipython.org/github/lmart999/GIS/blob/master/SF_GIS_Crime.ipynb), a GIS analysis of public crime data in SF, by [Lance Martin](https://github.com/lmart999).
### Psychology and Neuroscience
* [Cue Combination with Neural Populations](http://nbviewer.ipython.org/github/wtadler/cue-combination-with-neurons/blob/master/neural_cue_combination.ipynb) by [Will Adler](http://www.wtadler.com). Intuition and simulation for the theory (Ma *et al.*, 2006) that through probabilistic population codes, neurons can perform optimal cue combination with simple linear operations. Demonstrates that variance in cortical activity, rather than impairing sensory systems, is an adaptive mechanism to encode uncertainty in sensory measurements.
* [Modeling psychophysical data with non-linear functions](http://nbviewer.ipython.org/github/arokem/teach_optimization/blob/master/optimization.ipynb) by Ariel Rokem.
* [Visualizing mathematical models of brain cell connections](http://nbviewer.ipython.org/github/jonasnick/ReceptiveFields/blob/master/receptiveFields.ipynb). The effect of convolution of different receptive field functions and natural images is examined.
* [Python for Vision Research](http://nbviewer.ipython.org/github/gestaltrevision/python_for_visres/blob/master/index.ipynb). A three-day crash course for vision researchers in programming with Python, building experiments with [PsychoPy](http://psychopy.org/) and [psychopy_ext](http://psychopy_ext.klab.lt/), learning the fMRI multi-voxel pattern analysis with [PyMVPA](http://www.pymvpa.org/), and understading image processing in Python.
* [Loading and visualizing fMRI data](http://nbviewer.ipython.org/github/GaelVaroquaux/nilearn_course/blob/master/rendered_notebooks/1_Introduction.ipynb), part of the [Functional connectivity with NiLearn course](https://github.com/GaelVaroquaux/nilearn_course) by [Gaël Varoquaux](http://gael-varoquaux.info).
### Machine Learning, Statistics and Probability
* [An introduction to parallel machine learning with sklearn, joblib and IPython.parallel](http://nbviewer.ipython.org/gist/5115540/Model%20Selection%20for%20the%20Nystroem%20Method.ipynb), a notebook that accompanies this [slide deck](https://speakerdeck.com/ogrisel/parallel-and-large-scale-machine-learning-with-scikit-learn) by [Olivier Grisel](https://github.com/ogrisel).
* [A tutorial introduction to machine learning with sklearn](http://amueller.github.com/sklearn_tutorial), an IPython-based slide deck by [Andreas Mueller](https://github.com/amueller).
* [Introduction to Machine Learning in Python with scikit-learn](http://ipython-books.github.io/featured-04/) by [Cyrille Rossant](http://cyrille.rossant.net/), a free recipe from the [IPython Cookbook](http://ipython-books.github.io/cookbook/), a comprehensive guide to Python for Data Science.
* [An introduction to Predictive Modeling in Python](http://nbviewer.ipython.org/github/ogrisel/parallel_ml_tutorial/blob/master/solutions/01%20-%20An%20Introduction%20to%20Predictive%20Modeling%20in%20Python.ipynb), by [Olivier Grisel](https://github.com/ogrisel).
* [Face Recognition on a subset of the Labeled Faces in the Wild dataset](http://nbviewer.ipython.org/github/ogrisel/notebooks/blob/master/Labeled%20Faces%20in%20the%20Wild%20recognition.ipynb), by [Olivier Grisel](http://github.com/ogrisel).
* [An Introduction to Bayesian Methods for Multilevel Modeling](http://nbviewer.ipython.org/github/fonnesbeck/multilevel_modeling/blob/master/multilevel_modeling.ipynb), by [Chris Fonnesbeck](http://github.com/fonnesbeck).
* [Introduction to Bayesian Networks](http://nbviewer.ipython.org/github/kuitang/hackny-bayesnet/blob/master/hackNY%20Bayesian%20Network%20Demo.ipynb) by [Kui Tang](https://github.com/kuitang)
* [Bayesian data analysis with PyMC3](http://nbviewer.ipython.org/github/twiecki/pymc3_talk/blob/master/bayesian_pymc3.ipynb) by [Thomas Wiecki](https://github.com/twiecki).
* [A collection of examples for solving pattern classification problems](https://github.com/rasbt/pattern_classification), by [Sebastian Raschka](https://github.com/rasbt).
* [Introduction to Linear Regression using Python](http://nbviewer.ipython.org/github/justmarkham/DAT4/blob/master/notebooks/08_linear_regression.ipynb) by [Kevin Markham](https://github.com/justmarkham)
* [Machine learning in Python](http://nbviewer.ipython.org/github/jdwittenauer/ipython-notebooks/blob/master/ML-Exercise1.ipynb), a series based on Andrew Ng's Coursera class on machine learning. Part of a [larger collection of data science notebooks](https://github.com/jdwittenauer/ipython-notebooks) by [John Wittenauer](https://github.com/jdwittenauer).
* [Probability, Paradox, and the Reasonable Person Principle](http://nbviewer.ipython.org/url/norvig.com/ipython/Probability.ipynb), by Peter Norvig.
### Physics, Chemistry and Biology
* [Multibody dynamics and control with Python](http://www.moorepants.info/blog/npendulum.html) and the [notebook file](http://nbviewer.ipython.org/github/gilbertgede/idetc-2013-paper/blob/master/n-pendulum-control.ipynb) by Jason K. Moore.
* [Manipulation and display of chemical structures](http://nbviewer.ipython.org/gist/4316433), by [Greg Landrum](https://github.com/greglandrum), using rdkit.
* [The sound of Hydrogen](http://nbviewer.ipython.org/github/Carreau/posts/blob/master/07-the-sound-of-hydrogen.ipynb), visualizing and listening to the quantum-mechanical spectrum of Hydrogen. By [Matthias Bussonnier](http://github.com/Carreau).
* Particle physics at the Large Hadron Collider (LHC): using [ROOT in an LHCb masterclass](http://root.cern.ch/drupal/content/ipython-notebooks-and-root-0): [Notebook 1](http://nbviewer.ipython.org/github/mazurov/webfest2013/blob/master/notebooks/MasterClassD0-ex1.ipynb) and [Notebook 2](http://nbviewer.ipython.org/github/mazurov/webfest2013/blob/master/notebooks/MasterClassD0-ex2%2Cex3.ipynb) notebooks by Alexander Mazurov and Andrey Ustyuzhanin at CERN.
* [A Reaction-Diffusion Equation Solver in Python with Numpy](http://nbviewer.ipython.org/github/waltherg/notebooks/blob/master/2013-12-03-Crank_Nicolson.ipynb), a demonstration of how IPython notebooks can be used to discuss both the theory and implementation of numerical algorithms on one page, by [Georg Walther](https://github.com/waltherg).
* [Comparing different approaches to evolutionary simulations](http://nbviewer.ipython.org/url/www.slideviper.oquanta.info/test/simulation%20frameworks.ipynb). Also available [here](http://slideviewer.herokuapp.com/urls/github.com/yoavram/ipython-notebooks/raw/master/simulation%20frameworks.ipynb) to better visualization. The notebook was converted to a HTML presentation using an old nbconvert with the first developing implementation of `reveal` converter. By Yoav Ram.
### Economics and Finance
* [Replication of the highly-contentious analysis of economic growth by Reinhart and Rogoff](http://nbviewer.ipython.org/github/vincentarelbundock/Reinhart-Rogoff/blob/master/reinhart-rogoff.ipynb), by [Vincent Arel-Bundock](http://umich.edu/~varel), [full repo here](https://github.com/vincentarelbundock/Reinhart-Rogoff). This is based on the [widely-publicized critique of the original analysis done by Herndon, Ash, and Pollin](http://www.peri.umass.edu/236/hash/31e2ff374b6377b2ddec04deaa6388b1/publication/566).
* [fecon235 for Financial Economics](https://github.com/rsvp/fecon235) series of notebooks which examines time-series data for economics and finance. Easy API to freely access data from the Federal Reserve, SEC, CFTC, stock and futures exchanges. Thus research from older notebooks can be replicated, and updated using the most current data. For example, this notebook forecasts likely Fed policy for setting the [Fed Funds rate](https://git.io/fedfunds), but market sentiment across major asset classes is observable from the [CFTC Commitment of Traders Report](https://git.io/cotr). Major economics indicators are renormalized: for example, various measures of [inflation](https://git.io/infl), optionally with the forward-looking break-even rates derived from U.S. Treasury bonds. Other notebooks examine international markets: especially, gold and foreign exchange.
### Earth science and geo-spatial data
* [Exploring seafloor habitats: geographic analysis using IPython Notebook with GRASS & R](http://nbviewer.ipython.org/gist/7598354). This embeds a slideshow and a Web Spinning Globe (Cesium) in the notebook. By Massimo Di Stefano.
* [Geo-Spatial Data with IPython](http://nbviewer.ipython.org/github/mqlaql/geospatial-data/blob/master/Geospatial-Data-with-Python.ipynb). Tutorial by [Kelsey Jordahl](http://kjordahl.net/) from SciPy2013.
### Data visualization and plotting
* A [Notebook](http://nbviewer.ipython.org/github/plotly/python-user-guide/blob/master/s3_bubble-charts/s3_bubble-charts.ipynb) with an interactive Hans Rosling Gapminder bubble chart from [Plotly](https://plot.ly/api/Python).
* [Data and visualization integration via web based resources](http://tw.rpi.edu/media/2013/09/25/a48/The_Perfect_Storm_1991.html). Using NetCDF, Matplotlib, IPython Parallel and ffmpeg to generate video animation from time series of gridded data. By Massimo Di Stefano.
* [21 Interactive, D3 Plots from matplotlib, ggplot for Python,
prettyplotlib, Stack Overflow, and seaborn](http://nbviewer.ipython.org/gist/msund/7ac1203ded66fe8134cc).
* [Visualizing complex-valued functions with Matplotlib and Mayavi](http://nbviewer.ipython.org/github/empet/Math/blob/master/DomainColoring.ipynb), by [Emilia Petrisor](https://github.com/empet).
* [A D3 Viewer for Matplotlib Visualizations](http://jakevdp.github.io/blog/2013/12/19/a-d3-viewer-for-matplotlib/), different from above by not depending on Plot.ly account.
* [Bokeh is an interactive web visualization library for Python](http://nbviewer.ipython.org/github/damianavila/bokeh_overview/blob/master/Bokeh%20Overview.ipynb?create=1) (and other languages). It provides d3-like novel graphics, over large datasets, all without requiring any knowledge of Javascript. It also has a Matplotlib compatibility layer.
* [Winner of the 2014 E. Tufte Slope Graphs contest](http://nbviewer.ipython.org/gist/pascal-schetelat/8382651), by [Pascal Schetelat](https://github.com/pascal-schetelat). The original [contest info on Tufte's site](http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0003nk).
* [matta, d3.js-based visualizations in the IPython Notebook](http://nbviewer.ipython.org/github/carnby/matta/blob/master/examples/Basic%20Examples.ipynb), by [Eduardo Graells-Garrido](http://carnby.github.io/).
### Mathematics
* [Linear algebra with Cython](http://nbviewer.ipython.org/github/carljv/cython_testing/blob/master/cython_linalg.ipynb). A tutorial that styles the notebook differently to show that you can produce high-quality typography online with the Notebook. By Carl Vogel.
* [Exploring how smooth-looking functions can have very surprising derivatives even at low orders](http://nbviewer.ipython.org/url/finiterank.com/cuadernos/suavesylocas.ipynb), combining SymPy and matplotlib. By [Javier Moreno](http://finiterank.com).
* [A Collection of Applied Mathematics and Machine Learning Tutorials](http://sayilarvekuramlar.blogspot.co.uk/2015/12/matematik-ders-notlari.html) (in Turkish). By Burak Bayramli.
* [Function minimization with iminuit](http://nbviewer.ipython.org/github/iminuit/iminuit/blob/master/tutorial/tutorial.ipynb), an introductory companion to their [hard core tutorial](http://nbviewer.ipython.org/github/iminuit/iminuit/blob/master/tutorial/hard-core-tutorial.ipynb). By the [iminuit project](http://iminuit.github.io/iminuit).
* [The Discrete Cosine Transform](http://nbviewer.ipython.org/url/cs.marlboro.edu/courses/spring2014/information/code/dct/dct.ipynb), a brief explanation and illustration of the math behind the DCT and its role in the JPEG image format, by [Jim Mahoney](http://cs.marlboro.edu).
* [Chebfun in Python](http://nbviewer.ipython.org/gist/6724986), a demo of [PyChebfun](https://github.com/cswiercz/pychebfun), by [Olivier Verdier](http://www.olivierverdier.com). PyChebfun is a pure-python implementation of the celebrated [Chebfun package by Battles and Trefethen](http://people.maths.ox.ac.uk/trefethen/publication/PDF/2004_107.pdf).
* [The Matrix Exponential]
(http://nbviewer.ipython.org/github/sdrelton/matrix_function_notebooks/blob/master/TheMatrixExponential.ipynb), an introduction to the matrix exponential, its applications, and a list of available software in Python and MATLAB. By [Sam Relton](http://www.maths.manchester.ac.uk/~srelton/).
* [Fractals, complex numbers, and your imagination](http://nbviewer.ipython.org/github/cfangmeier/ipython_notebooks/blob/master/Imagination.ipynb), by [Caleb Fangmeier](https://github.com/cfangmeier).
* [A SymPy tutorial](http://nbviewer.ipython.org/url/www.inp.nsk.su/~grozin/python/sympy.ipynb), by [Andrey Grozin](http://www.inp.nsk.su/~grozin/).
### Signal and Sound Processing
* [Simulation of Delta Sigma modulators in Python](http://nbviewer.ipython.org/github/ggventurini/python-deltasigma/blob/master/examples/dsdemo1.ipynb) with [deltasigma](https://github.com/ggventurini/python-deltasigma), Python port of of Richard Schreier's *excellent* [MATLAB Delta Sigma Toolbox](http://www.mathworks.com/matlabcentral/fileexchange/19-delta-sigma-toolbox), by [Giuseppe Venturini](https://github.com/ggventurini). Several demonstrative notebooks on the package [README](https://github.com/ggventurini/python-deltasigma/blob/master/README.md).
* [PyOracle: Automatic analysis of musical structure](http://nbviewer.ipython.org/urls/bitbucket.org/pucktronix/pyoracle/raw/d046b2bcf473503fa356094cfe4cff774d1aaefc/270D.ipynb), by [Greg Surges](http://gregsurges.com).
* [A Gallery of SciPy's Window Functions for quick visual inspection and comparison] (http://nbviewer.ipython.org/urls/gist.githubusercontent.com/jaidevd/b7d865f7f4b237ab5181/raw/30bc8f998bf8f924b56b32ce10acce125656ed7c/scipy_window_gallery.ipynb)
by [Jaidev Deshpande](http://twitter.com/jaidevd)
### Natural Language Processing
* [Python Programming for the Humanities](http://fbkarsdorp.github.io/python-course/) by Folgert Karsdorp & Maarten van Gompel.
### Pandas for data analysis
Note that in the 'collections' section above there are also pandas-related links, such as the one for an [11-lesson tutorial](https://bitbucket.org/hrojas/learn-pandas).
* [A 10-minute whirlwind tour of pandas](http://nbviewer.ipython.org/gist/wesm/4757075/PandasTour.ipynb), this is the notebook accompanying a [video presentation](http://vimeo.com/59324550) by Wes McKinney, author of Pandas and the [Python for Data Analysis](http://www.amazon.com/Python-Data-Analysis-Wes-McKinney/dp/1449319793) book.
* [Time-series analysis with Pandas](http://nbviewer.ipython.org/github/changhiskhan/talks/blob/master/pydata2012/pandas_timeseries.ipynb).
* [Financial data analysis with Pandas](http://nbviewer.ipython.org/gist/3962843).
* [Clustering of smartphone sensor data for human activity detection using pandas and scipy](http://nbviewer.ipython.org/github/herrfz/dataanalysis/blob/master/week4/clustering_example.ipynb), part of Coursera data analysis course, done in Python ([repo](https://github.com/herrfz/dataanalysis)).
* [Log analysis with Pandas](http://nbviewer.ipython.org/url/taaviburns.ca/presentations/log_analysis_with_pandas/nb/5-Scatterplots.ipynb), part of a [group presented at PyConCa 2012](http://taaviburns.ca/presentations/log_analysis_with_pandas/) by Taavi Burns.
* [Analyzing and visualizing sun spot data with Pandas](http://nbviewer.ipython.org/gist/4569783), by [Josh Hemann](https://github.com/jhemann). An enlightening discussion of how naive plotting choices subtly influence our interpretation of data.
* [Advanced analysis of Apache logs](http://nbviewer.ipython.org/github/koldunovn/nk_public_notebooks/blob/master/Apache_log.ipynb), by [Nikolay Koldunov](https://github.com/koldunovn).
* [Statistical Data Analysis in Python] (https://github.com/fonnesbeck/statistical-analysis-python-tutorial), by [Christopher Fonnesbeck](https://github.com/fonnesbeck/), SciPy 2013. Companion videos [1](https://www.youtube.com/watch?v=DXPwSiRTxYY), [2](https://www.youtube.com/watch?v=TGEBpzJUxdI), [3](https://www.youtube.com/watch?v=YZDtBEEZuAk), [4](https://www.youtube.com/watch?v=5_rcdhBXD-0)
## General Python Programming
* [Learning to code with Python](http://nbviewer.ipython.org/urls/bitbucket.org/amjoconn/watpy-learning-to-code-with-python/raw/3441274a54c7ff6ff3e37285aafcbbd8cb4774f0/notebook/Learn%20to%20Code%20with%20Python.ipynb), part of an [introduction to Python](https://bitbucket.org/amjoconn/watpy-learning-to-code-with-python/src) from the [Waterloo Python users group](http://watpy.ca/blog/post/learn-code-python-review-feb-2013).
* [Python Descriptors Demystified](http://nbviewer.ipython.org/gist/ChrisBeaumont/5758381/descriptor_writeup.ipynb), an in-depth discussion of the descriptor protocol in Python, by [Chris Beaumont](http://chrisbeaumont.org).
* [A collection of not so obvious Python stuff you should know!](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/not_so_obvious_python_stuff.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt).
* [Key differences between Python 2.7.x and Python 3.x](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/key_differences_between_python_2_and_3.ipynb), by [Sebastian Raschka](https://github.com/rasbt).
* [A beginner's guide to Python's namespaces, scope resolution, and the LEGB rule](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/scope_resolution_legb_rule.ipynb?create=1), by [Sebastian Raschka](https://github.com/rasbt).
* [Sorting CSV files using the Python csv module](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/sorting_csvs.ipynb), by [Sebastian Raschka](https://github.com/rasbt).
* Python 3 OOP series by [Leonardo Giordani](https://github.com/lgiordani): [Part 1: Objects and types](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_1__Objects_and_types.ipynb), [Part 2: Classes and members](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_2__Classes_and_members.ipynb), [Part 3: Delegation - composition and inheritance](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_3__Delegation__composition_and_inheritance.ipynb), [Part 4: Polymorphism](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_4__Polymorphism.ipynb), [Part 5: Metaclasses](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_5__Metaclasses.ipynb), [Part 6: Abstract Base Classes](http://nbviewer.ipython.org/github/lgiordani/blog_source/blob/master/pelican/content/notebooks/Python_3_OOP_Part_6__Abstract_Base_Classes.ipynb)
## Notebooks in languages other than Python
These are notebooks that use [one of the IPython kernels for other languages](IPython kernels for other languages):
### Julia
The IPython protocols to communicate between kernels and clients are language agnostic, and other programming language communities have started to build support for this protocol in their language. The Julia team has created [IJulia](https://github.com/JuliaLang/IJulia.jl), and these are some Julia notebooks:
* [Fractals 3 ways](http://nbviewer.ipython.org/url/beowulf.csail.mit.edu/18.337/fractals.ipynb), by [Jeff Bezanson](https://github.com/JeffBezanson).
* [The Design Impact of Multiple Dispatch](http://nbviewer.ipython.org/gist/StefanKarpinski/b8fe9dbb36c1427b9f22), a detailed explanation of Julia's multiple dispatch design, by [Stefan Karpinski](https://github.com/StefanKarpinski).
* A [tutorial](http://nbviewer.ipython.org/gist/7551139) on making interactive graphs with [Plotly](https://plot.ly/) and Julia.
* [Numerical tours in Julia](http://www.numerical-tours.com/julia/)
* [Functional Geometry](http://nbviewer.ipython.org/github/shashi/ijulia-notebooks/blob/master/funcgeo/Functional%20Geometry.ipynb) by [Shashi Gowda](https://github.com/shashi)
* [JuliaOpt notebooks](http://nbviewer.ipython.org/github/JuliaOpt/juliaopt-notebooks/tree/master/notebooks/), a collection of optimization-related notebooks.
* Coursework using IJulia notebooks:
- [Métodos Numéricos Avanzados (2015-2)](https://github.com/dpsanders/MetodosNumericosAvanzados), Luis Benet and David P. Sanders
- [Métodos Monte Carlo](https://github.com/dpsanders/metodos-monte-carlo), David Sanders
- [Linear Partial Differential Equations: Analysis and Numerics](http://math.mit.edu/~stevenj/18.303/), Steven G. Johnson
- [Julia tutorial for Computational Molecular Biology](http://cs.brown.edu/courses/csci1810/Julia_Tutorial.html), Younhun Kim and Matthew Reyna
* Other collections of IJulia notebooks:
- [Jiahao Chen](http://jiahao.github.io/code/)
- [Christoph Ortner](https://homepages.warwick.ac.uk/staff/C.Ortner/index.php?page=julia)
- [Crossing Language Barriers with Julia, Scipy, and IPython](https://github.com/stevengj/Julia-EuroSciPy14), presented at EuroSciPy '14 by Steven G. Johnson.
### Haskell
There exists a Haskell kernel for IPython in the [IHaskell project](https://github.com/gibiansky/ihaskell).
* [IHaskell Demo Notebook](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/IHaskell.ipynb)
* [Homophone reduction](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/Homophones.ipynb), a solution to a cute problem involving treating English letters as generators of a large group.
* [Gradient descent typeclass](http://nbviewer.ipython.org/github/gibiansky/IHaskell/blob/master/notebooks/Gradient-Descent.ipynb), a look at how arbitrary gradient descent algorithms can be represented with a typeclass.
### OCaml
[iocaml](https://github.com/andrewray/iocaml) is an OCaml kernel for IPython
* [H.261 Video Decoding in OCaml](https://andrewray.github.io/iocamljs/oh261.html)
* [OCaml implementation of the 2048 game](http://gazagnaire.org/fuconf14/)
### Ruby
Similar to the Julia kernel there exists also a [Ruby kernel](https://github.com/SciRuby/iruby) for IPython.
* [IRuby Demo Notebook](http://nbviewer.ipython.org/github/SciRuby/sciruby-notebooks/blob/master/getting_started.ipynb)
* [SciRuby Notebooks](https://github.com/SciRuby/sciruby-notebooks)
The interactive plotting library [Nyaplot](https://github.com/domitry/nyaplot) has some case studies using IRuby:
* [War expenditure per GDP](http://nbviewer.ipython.org/github/domitry/nyaplot/blob/master/examples/notebook/Mapnya.ipynb#Case2-:Fill-countries-in-different-colors)
* [Finding shape consensus among multiple geometrical polygons](http://nbviewer.ipython.org/gist/mgiraldo/a68b53175ce5892531bc)
### Perl
* An example showcasing full use of the [display protocol](http://nbviewer.ipython.org/github/zmughal/zmughal-iperl-notebooks/blob/master/IPerl-demos/20150209_IPerl_display_demo.ipynb) with the IPerl kernel.
## Miscellaneous topics about doing various things *with* the Notebook itself
* [Blogging With IPython in Blogger](http://nbviewer.ipython.org/github/fperez/blog/blob/master/120907-Blogging%20with%20the%20IPython%20Notebook.ipynb), also available in [blog post form](http://blog.fperez.org/2012/09/blogging-with-ipython-notebook.html), [full repo here](https://github.com/fperez/blog). By Fernando Perez.
* [Blogging With IPython in Octopress](http://nbviewer.ipython.org/urls/github.com/jakevdp/jakevdp.github.com/raw/master/downloads/notebooks/nb_in_octopress.ipynb), by Jake van der Plas and available as a [blog post](http://jakevdp.github.com/blog/2012/10/04/blogging-with-ipython/). Other [notebooks by Jake](https://github.com/jakevdp/jakevdp.github.com/tree/master/downloads/notebooks) contain many more great examples of doing interesting work with the scientific Python stack.
* [Blogging With IPython in Nikola](http://nbviewer.ipython.org/url/www.damian.oquanta.info/posts/blogging-with-nikola-and-ipython.ipynb), also available in [blog post form](http://www.damian.oquanta.info/posts/blogging-with-nikola-and-ipython.html) by Damián Avila.
* [Custom CSS control of the notebook](http://nbviewer.ipython.org/github/Carreau/posts/blob/master/Blog1.ipynb), this is part of a [blog repo](https://github.com/Carreau/posts) by Matthias Bussonnier.
* [IPython display hookery: tools to help display visual output from various sources](http://nbviewer.ipython.org/gist/5162445), a gist by [@deeplook](https://github.com/deeplook).
* [Importing IPython Notebooks as Modules](http://nbviewer.ipython.org/gist/6011986) by [Min RK](http://github.com/minrk).
## Reproducible academic publications
This section contains academic papers that have been published in the peer-reviewed literature or pre-print sites such as the [ArXiv](http://arxiv.org) that include one or more notebooks that enable (even if only partially) readers to reproduce the results of the publication. If you include a publication here, please link to the journal article as well as providing the nbviewer notebook link (and any other relevant resources associated with the paper).
1. [Reply to 'Influence of cosmic ray variability on the monsoon rainfall and temperature': a false-positive in the field of solar-terrestrial research](http://arxiv.org/abs/1502.00505) by [Benjamin Laken](http://www.benlaken.com), 2015. Reviewed article will appear in JASTP. The [IPython notebook](http://nbviewer.ipython.org/github/benlaken/Comment_BadruddinAslam2014/blob/master/Monsoon_analysis.ipynb) reproduces the full analysis and figures exactly as they appear in the article, and is available on Github: link via [figshare](http://figshare.com/articles/Comment_on_Badruddin_amp_Aslam_2014_/1299413).
1. [The probability of improvement in Fisher's geometric model: a probabilistic approach](http://dx.doi.org/10.1016/j.tpb.2014.10.004), by [Yoav Ram](http://www.yoavram.com/) and [Lilach Hadany](https://sites.google.com/site/hadanylab/). (Theoretical Population Biology, 2014). An [IPython notebook](http://nbviewer.ipython.org/url/www.sciencedirect.com/science/MiamiMultiMediaURL/1-s2.0-S0040580914000811/1-s2.0-S0040580914000811-mmc1.txt/272364/FULL/S0040580914000811/471cf02085a52c248dc76ae65ad4409d/mmc1.txt), allowing figure reproduction, was deposited as a [supplementry file](http://www.sciencedirect.com/science/MiamiMultiMediaURL/1-s2.0-S0040580914000811/1-s2.0-S0040580914000811-mmc1.txt/272364/FULL/S0040580914000811/471cf02085a52c248dc76ae65ad4409d/mmc1.txt).
1. [Stress-induced mutagenesis and complex adaptation](http://rspb.royalsocietypublishing.org/content/281/1792/20141025.abstract), by [Yoav Ram](http://www.yoavram.com/) and [Lilach Hadany](https://sites.google.com/site/hadanylab/) (Proceedings B, 2014). An [IPython notebook](https://github.com/yoavram/ruggedsim/blob/master/manuscript/supplementry.ipynb), allowing figures reproduction, was deposited as a [supplementry file](http://rspb.royalsocietypublishing.org/content/suppl/2014/08/19/rspb.2014.1025.DC1).
1. [Automatic segmentation of odor maps in the mouse olfactory bulb using regularized non-negative matrix factorization](http://www.sciencedirect.com/science/article/pii/S1053811914003103), by J. Soelter et al. (Neuroimage 2014, Open Access). The [notebook](http://nbviewer.ipython.org/github/jansoe/FUImaging/blob/master/examples/IOSsegmentation/regNMF.ipynb) allows to reproduce most figures from the paper and provides a deeper look at the data. The [full code repository](https://github.com/jansoe/FUImaging/tree/Neuroimage2014) is also available.
1. [Multi-tiered genomic analysis of head and neck cancer ties TP53 mutation to 3p loss, by A. Gross et al. (Nature Genetics 2014)](http://www.nature.com/ng/journal/vaop/ncurrent/full/ng.3051.html). The [full collection of notebooks to replicate the results](https://github.com/theandygross/TCGA/tree/master/Analysis_Notebooks#guide-to-running).
1. [powerlaw: a Python package for analysis of heavy-tailed distributions, by J. Alstott et al.](https://code.google.com/p/powerlaw/). [Notebook of examples in manuscript](http://nbviewer.ipython.org/gist/19fcdd6a4ba400ce8de2), [ArXiv link](http://arxiv.org/abs/1305.0215) and [project repository](https://github.com/jeffalstott/powerlaw).
1. [Collaborative cloud-enabled tools allow rapid, reproducible biological insights, by B. Ragan-Kelley et al.](http://www.nature.com/ismej/journal/v7/n3/full/ismej2012123a.html). The [main notebook](http://nbviewer.ipython.org/gist/3693491/cloud_demo_complete.ipynb), the [full collection of related notebooks](http://nbviewer.ipython.org/gist/3693491) and the [companion site](http://qiime.org/home_static/nih-cloud-apr2012) with the Amazon AMI information for reproducing the full paper.
1. [A Reference-Free Algorithm for Computational Normalization of Shotgun Sequencing Data, by C.T. Brown et al.](http://ged.msu.edu/papers/2012-diginorm). [Full notebook](http://nbviewer.ipython.org/urls/github.com/ged-lab/2012-paper-diginorm/raw/master/notebook/diginorm.ipynb), [ArXiv link](http://arxiv.org/abs/1203.4802) and [project repository](https://github.com/ged-lab/2012-paper-diginorm).
1. [The kinematics of the Local Group in a cosmological context](http://arxiv.org/abs/1303.2690) by [J.E. Forero-Romero et al.](http://wwwprof.uniandes.edu.co/~je.forero/). The [Full notebook](http://nbviewer.ipython.org/github/forero/LG_Kinematics/blob/master/code/main_analysis.ipynb) and also all the data in a [github repo](https://github.com/forero/LG_Kinematics).
1. [Warming Ocean Threatens Sea Life](http://www.scientificamerican.com/article.cfm?id=warming-ocean-threatens-sea-life), an article in Scientific American [backed by a notebook for its main plot](http://nbviewer.ipython.org/github/robertodealmeida/notebooks/blob/master/scientific_american/Scientific%20American%20graph.ipynb). By [Roberto de Almeida](https://github.com/robertodealmeida) from [MarinExplore](https://marinexplore.com).
1. [Extrapolating Weak Selection in Evolutionary Games](http://nbviewer.ipython.org/github/juliangarcia/ews/blob/master/notebook.ipynb), by Wu, García, Hauert and Traulsen. [PLOS Comp Bio paper](http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1003381) and [Figshare link](http://figshare.com/articles/Extrapolating_weak_selection_in_evolutionary_games_source_code/814470).
1. [Using neural networks to estimate redshift distributions. An application to CFHTLenS]
(http://nbviewer.ipython.org/urls/bitbucket.org/christopher_bonnett/nn_notebook/raw/5e69b55193a229cb2076a2f18e43b45c56e317e0/T-800.ipynb)
by Christopher Bonnett [paper](http://arxiv.org/abs/1312.1287)(submitted to MNRAS)
1. [Mechanisms for stable, robust, and adaptive development of orientation maps in the primary visual cortex](http://dx.doi.org/10.1523/JNEUROSCI.1037-13.2013) by Jean-Luc R. Stevens, Judith S. Law, Jan Antolik, and James A. Bednar. Journal of Neuroscience, 33:15747-15766, 2013. [Notebook1]
(https://ioam.github.io/topographica/_static/gcal_notebook.html), [Notebook2](https://ioam.github.io/topographica/_static/stevens_jn13_notebook.html).
1. [Accelerated Randomized Benchmarking](http://nbviewer.ipython.org/github/cgranade/accelerated-randomized-benchmarking/blob/master/src/model_testing.ipynb), by [Christopher Granade](http://www.cgranade.com/), [Christopher Ferrie](https://sites.google.com/site/csferrie/) and D. G. Cory. [New Journal of Physics **17** 013042 (2015)](http://iopscience.iop.org/article/10.1088/1367-2630/17/1/013042/meta;jsessionid=1F48CDD7C7D7849B0777C495ED1551CC.c1), [arXiv](http://arxiv.org/abs/1404.5275), [GitHub repo](https://github.com/cgranade/accelerated-randomized-benchmarking).
1. [Dynamics and associations of microbial community types across the human body](http://dx.doi.org/10.1038/nature13178), by Tao Ding & Patrick D. Schloss. [Notebook replicating results](http://nbviewer.ipython.org/gist/pschloss/9815766/notebook.ipynb).
1. [Variations in submarine channel sinuosity as a function of latitude and slope](http://nbviewer.ipython.org/gist/zsylvester/6040d0015b9b907bc788), by Sylvester, Z., Pirmez, C., Cantelli, A., & Jobe, Z. R.
1. [Frontoparietal representations of task context support the flexible control of goal directed cognition](http://www.jneurosci.org/content/34/32/10743.short), by M.L. Waskom, D. Kumaran, A.M. Gordon, J. Rissman, & A.D. Wagner. [Github repository](https://github.com/WagnerLabPapers/Waskom_JNeurosci_2014) | [Main notebook](http://nbviewer.ipython.org/github/WagnerLabPapers/Waskom_JNeurosci_2014/blob/master/Behavioral_and_Decoding_Analyses.ipynb)
1. [pyparty: Intuitive Particle Processing in Python](http://openresearchsoftware.metajnl.com/article/view/jors.bh), Adam Hughes [Notebook to Generate the Published Figures](http://nbviewer.ipython.org/github/hugadams/pyparty/blob/master/examples/Notebooks/JORS_data.ipynb?create=1) | Also, check out the [pyparty tutorial notebooks](https://github.com/hugadams/pyparty).
1. [Indication of family-specific DNA methylation patterns in developing oysters](http://biorxiv.org/content/early/2014/12/16/012831), Claire E. Olson, Steven B. Roberts
doi: http://dx.doi.org/10.1101/012831. [Notebook to generate results in the paper](http://nbviewer.ipython.org/github/che625/olson-ms-nb/blob/master/BiGo_dev.ipynb).
1. [Parallel Prefix Polymorphism Permits Parallelization, Presentation & Proof](http://conferences.computer.org/hptcdl/2014/papers/7020a047.pdf), [Jiahao Chen](http://jiahao.github.io) and [Alan Edelman](http://www-math.mit.edu/~edelman/), HPTCDL'14. [Website](http://jiahao.github.io/parallel-prefix) and [notebook](https://github.com/jiahao/ijulia-notebooks/blob/master/2014-08-06-parallel-prefix.ipynb)
1. [Transcriptome Sequencing Reveals Potential Mechanism of Cryptic 3 Splice Site Selection in *SF3B1*-mutated Cancers](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004105) by [Christopher DeBoever](http://cdeboever3.github.io/) et al. There are several [notebooks](https://github.com/cdeboever3/deboever-sf3b1-2015/tree/master/notebooks) to replicate results and make figures.
1. [A Workflow for Characterizing Nanoparticle Monolayers for Biosensors: Machine Learning on Real and Artificial SEM Images](https://peerj.com/preprints/671/), Adam Hughes, Zhaowen Liu, Maryam Raftari, Mark. E Reeves. Notebooks are linked in Table 1 in the text.
1. [AtomPy: An Open Atomic Data Curation Environment for Astrophysical Applications](http://www.mdpi.com/2218-2004/2/2/123), by C. Mendoza, J. Boswell, D. Ajoku, M. Bautista.
1. [Visualizing 4-Dimensional Asteroids](http://blogs.scientificamerican.com/sa-visual/2014/09/16/visualizing-4-dimensional-asteroids/), in Scientific American (by Jake VanderPlas)
1. [Challenges and opportunities in understanding microbial communities with metagenome assembly](http://journal.frontiersin.org/article/10.3389/fmicb.2015.00678/abstract), [accompanied by IPython Notebook tutorial](http://nbviewer.ipython.org/github/germs-lab/frontiers-review-2015/blob/master/frontiers-nb-2015.ipynb), by [Adina Howe](http://germslab.org) and Patrick Chain.
## Data-driven journalism
* [The Need for Openness in Data Journalism](http://nbviewer.ipython.org/github/brianckeegan/Bechdel/blob/master/Bechdel_test.ipynb), by [Brian Keegan](http://www.brianckeegan.com).
* [St. Louis County Segregation Analysis](https://github.com/BuzzFeedNews/2014-08-st-louis-county-segregation) , analysis for the article [The Ferguson Area Is Even More Segregated Than You Probably Guessed](http://www.buzzfeed.com/jsvine/the-ferguson-area-is-even-more-segregated-than-you-thought) by [Jeremy Singer-Vine](https://twitter.com/jsvine).
## Whimsical notebooks
* [XKCD-styled plots created with Matplotlib](http://nbviewer.ipython.org/url/jakevdp.github.com/downloads/notebooks/XKCD_plots.ipynb). Here is the [blog post version](http://jakevdp.github.com/blog/2012/10/07/xkcd-style-plots-in-matplotlib/) with discussion. By Jake van der Plas.
* [Van Gogh's Starry Night with ipythonblocks](http://nbviewer.ipython.org/github/jiffyclub/ipythonblocks/blob/master/demos/starry_night_to_text.ipynb), part of Matt Davis' [ipythonblocks](https://github.com/jiffyclub/ipythonblocks). This is a teaching tool for use with the IPython notebook that provides visual elements to understand programming concepts.
* [Conway's Game of Life](http://nbviewer.ipython.org/gist/3778422). Interesting use of convolution operation to calculate the next state of game board, instead of obvious find neighbors and filter the board for next state.
* [pynguins](https://gist.github.com/denfromufa/9a5e1fdeaf611dc60ea8). Using jupyter notebook, python, and numpy to solve Board Game "Penguins on Ice".
* ["People plots"](http://nbviewer.ipython.org/gist/4544012), stick figures generated with matplotlib.
* [Reveal converter mini-tutorial](http://nbviewer.ipython.org/url/www.damian.oquanta.info/posts/reveal-converter-mini-tutorial.ipynb), also available in [blog post form](http://www.damian.oquanta.info/posts/reveal-converter-mini-tutorial.html). Do you want to make static html/css slideshow straight from the IPython notebook? OK, now you can do it with the *reveal* converter (nbconvert). [Demo](http://www.slideviper.oquanta.info/tutorial/slideshow_tutorial_slides.html) by Damián Avila.
* [Personal IPython Weight Notebook] (http://nbviewer.ipython.org/gist/9769238). Plot your loss of weight with prognosis and motivation features.
* [Streaming Double Pendulum Simulation in IPython NB](http://nbviewer.ipython.org/github/plotly/python-user-guide/blob/master/s7_streaming/s7_streaming.ipynb).
* [Porque Charles Xavier debe cambiar a Cerebro por Python](http://nbviewer.ipython.org/github/mshopper/aurora/blob/master/Aurora.ipynb), a study in data and gender in the Marvel comics universe, by [Mai Giménez](http://twitter.com/adahopper) and [Angela Rivera](http://twitter.com/ghilbrae).
* [Functional Geometry: a deconstruction of the MC Escher woodcut Square Limit](http://nbviewer.ipython.org/github/shashi/ijulia-notebooks/blob/master/funcgeo/Functional%20Geometry.ipynb), an IJulia notebook by [Shashi Gowda](http://shashi.github.io/).
## Videos of IPython being used in the wild
Of course the first thing you might try is searching for videos about IPython (1900 or so by last count on Youtube) but there are demonstrations of other applications using the power of IPython but are not mentioned is the descriptions. Below are a few such:
* [Video](http://www.youtube.com/watch?v=Nc16qeGBtMU) on how to learn Python featuring IPython as the platform of choice for learning!
* [This video](http://www.youtube.com/watch?v=4ONBVNm3isI) shows IPython being used in the [scikit-learn](http://scikit-learn.org) project
* He doesn't show IPython in use but his IPython sticker is clear for the entire video: [Planning and Tending the Garden: The Future of Early Childhood Python Education](http://www.youtube.com/watch?v=op61s-QHryk)
* [Wes McKinney's speech](http://youtu.be/qbYYamU42Sw?t=5m9s) on Python and data analysis features IPython as does his book [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do)
* [This video](https://www.youtube.com/watch?v=zG8FYPFU9n4) shows Plotly and IPython in use at a Montreal Python meetup.

104
userguide/zh/notebook/nb.md Normal file
View File

@ -0,0 +1,104 @@
## 使用 Notebook
进入 Dashboard , 选择 ```New - Python3```, 创建 Python3 Notebook, 会
进入 Python3 Notebook 交互编程环境,并生成一个 ```untitled.ipynb``` 的
IPython Notebook文件。
本节介绍了一个基本的[Hello World](#hello-world), 一个[机器学习并可视化结
果](#一个机器学习并可视化结果的示例), 一个[完整教程](#完整教程) 的链接。
### Hello World ###
在 Cell 中输入
```python
print("Hello World")
```
执行后得到输出
```
Hello World
```
### 一个机器学习并可视化结果的示例
绘图需要加载 matplotlib. 在 Cell 中输入并执行
```python
%matplotlib inline
```
或者
```python
%matplotlib notebook
```
即可开启在浏览器中绘图的支持。
在后面的 Cell 中输入并执行下面关于机器学习的代码
```python
# import package
import numpy as np
import matplotlib.pyplot as plt
from sklearn import linear_model, datasets
# load data : we only use target==0 and target==1 (2 types classify) and feature 0 and feature 2 ()
iris = datasets.load_iris()
X = iris.data[iris.target!=2][:, [0,2]]
Y = iris.target[iris.target!=2]
h = .02 # step size in the mesh
logreg = linear_model.LogisticRegression(C=1e5)
logreg.fit(X, Y)
# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, m_max]x[y_min, y_max].
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = logreg.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
#plt.figure(1, figsize=(4, 3))
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=Y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())
plt.savefig("learn.svg")
```
稍后可看到如下输出
<img src="../images/sklearn1.png">
当前目录下同时会生成矢量图形文件 ```learn.svg```,可以在 Dashboard 中打
开, 也可以直接在 Cell 中打开
```
SVG("learn.svg")
```
### 完整教程 ###
在IPython Notebook中 能做很多事情,如显示本地磁盘中的各种图形文件、视频
文件、HTML文件加载一个外部网站显示 LaTeX 公式等,完整介绍和教程请见
[这里](http://nbviewer.jupyter.org/github/ipython/ipython/blob/4.0.x/examples/IPython%20Kernel/Index.ipynb).
[nbviewer](http://nbviewer.jupyter.org) 同样提供了很多其他应用示例。

View File

@ -0,0 +1,821 @@
## Python 软件包 ##
Docklet 基础镜像中 Python3 相关的软件包列表:
<table>
<caption></caption>
<thead>
<tr>
<th>Python Software</th>
<th>License</th>
<th>Summary</th>
<th class="center-txt">Note</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="middle">
<a target="_blank" href="http://matplotlib.org/basemap/">basemap</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://matplotlib.sourceforge.net/users/license.html">PSF</a></td>
<td valign="middle">Plot data on map projections with Matplotlib</td>
<td class="center-txt">
Map
</td>
</tr><tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/Blosc/bcolz">bcolz</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt">BSD</a></td>
<td valign="middle">Columnar and compressed data containers.</td>
<td class="center-txt">
Data
</td>
</tr><tr>
<td valign="middle">
<a target="_blank" href="https://github.com/pyca/bcrypt">bcrypt</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://pypi.python.org/pypi/bcrypt/">Apache</a></td>
<td valign="middle">Modern password hashing for your software and your servers</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.crummy.com/software/BeautifulSoup/">BeautifulSoup4</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.crummy.com/software/BeautifulSoup/">MIT</a></td>
<td valign="middle">Sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://biopython.org/wiki/Main_Page">biopython</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.biopython.org/DIST/LICENSE">(as-is) BSD-like</a></td>
<td valign="middle">Tools for biological computation</td>
<td class="center-txt">
Biological
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/ContinuumIO/blaze">blaze</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Data migration utilities</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://bokeh.pydata.org/">bokeh</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Statistical and novel interactive HTML plots for Python</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://scitools.org.uk/cartopy/docs/latest/index.html">cartopy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://scitools.org.uk/cartopy/docs/latest/index.html">LGPL</a></td>
<td valign="middle">A Python package designed to make drawing maps for data analysis and visualisation as easy as possible</td>
<td class="center-txt">
Map
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/enthought/chaco">chaco</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/enthought/chaco/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Library for generating interactive 2D plots</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://www.cheetahtemplate.org/">cheetah</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Template engine and code generation tool</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://nedbatchelder.com/code/modules/coverage.html">coverage</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Code coverage measurement for Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/cryptography">cryptography</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">cryptography is a package which provides cryptographic recipes and primitives to Python developers.</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://cython.org/">Cython</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/cython/cython/blob/master/LICENSE.txt">Apache</a></td>
<td valign="middle">Python-based language for writing C extensions for Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://docutils.sourceforge.net">docutils</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://svn.python.org/projects/external/docutils-0.5/COPYING.txt">public-domain with exceptions (BSD, PSF)</a></td>
<td valign="middle">Documentation utilities</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/flask">flask</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A lightweight web application framework based on Werkzeug, Jinja2 and good intentions.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://github.com/mitsuhiko/flask-babel">Flask-Babel</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://github.com/mitsuhiko/flask-babel">BSD</a></td>
<td valign="middle">Adds i18n/l10n support to Flask applications</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.gevent.org">gevent</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Python network library using greenlet and libevent for easy and scalable concurrency</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://bitbucket.org/Jeffrey/gevent-websocket">gevent_websocket</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a></td>
<td valign="middle">Websocket handler for the gevent pywsgi server, a Python network library</td>
<td class="center-txt">
Network
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/yhat/ggplot/">ggplot</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/yhat/ggplot/">BSD</a></td>
<td valign="middle">ggplot for python</td>
<td class="center-txt">
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.h5py.org/">h5py</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.h5py.org/docs/meta/licenses.html">BSD</a></td>
<td valign="middle">Python interface to the HDF library</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://jinja.pocoo.org/2/">Jinja2</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://dev.pocoo.org/hg/jinja2-main/raw-file/744e087d24e9/LICENSE">BSD</a></td>
<td valign="middle">Template engine</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://jupyter.org">jupyter</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://jupyter.org">BSD</a></td>
<td valign="middle">Jupyter metapackage. Install all the Jupyter components in one go.</td>
<td class="center-txt">
Web
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://lmfit.github.io/lmfit-py/">lmfit</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://lmfit.github.io/lmfit-py/">BSD</a></td>
<td valign="middle">Least-Squares Minimization with Bounds and Constraints</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://lxml.de/">lxml</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://lxml.de/index.html#license">BSD</a></td>
<td valign="middle">XML/XSLT library with bindings to libxml2/libxslt</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/M2Crypto">M2Crypto</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">M2Crypto: A Python crypto and SSL toolkit</td>
<td class="center-txt">
Crypt
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://matplotlib.sourceforge.net/">matplotlib</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://matplotlib.org/users/license.html">PSF</a></td>
<td valign="middle">Interactive 2D plotting library</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://mdp-toolkit.sourceforge.net/">MDP</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://mdp-toolkit.sourceforge.net/install.html">BSD</a></td>
<td valign="middle">Modular toolkit for Data Processing (MDP)</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/memory_profiler">memory_profiler</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://pypi.python.org/pypi/memory_profiler">BSD</a></td>
<td valign="middle">A module for monitoring memory usage of a python program</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://msgpack.org/">msgpack</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Efficient binary serialization library</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/mrocklin/multipledispatch/">multipledispatch</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/mrocklin/multipledispatch/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Multiple dispatch in Python.</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://networkx.lanl.gov/">networkx</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://networkx.lanl.gov/reference/legal.html">BSD</a></td>
<td valign="middle">Create, manipulate, and analyze graphs and networks</td>
<td class="center-txt">
Graph
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.nltk.org/">nltk</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Suite of tools for symbolic and statistical natural language processing</td>
<td class="center-txt">
NLP
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/numexpr">numexpr</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Fast evaluation of array expressions</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://numpy.scipy.org/">numpy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">General-purpose multi-dimensional array-processing and math library</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://bitbucket.org/ericgazoni/openpyxl/wiki/Home">openpyxl</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT/Expat</a></td>
<td valign="middle">Read and and write Excel OpenXML files</td>
<td class="center-txt">
Excel
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pandas.pydata.org/">pandas</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Data manipulation and analysis library</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/Paste">paste</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Tools for using a Web Server Gateway Interface stack</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/patsy">patsy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Package for describing statistical models and building design matrices</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pep8/">pep8</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://code.enthought.com/license.php">BSD</a></td>
<td valign="middle">Python style guide checker</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pexpect.readthedocs.org/">pexpect</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://pexpect.readthedocs.org/">ISC license</a></td>
<td valign="middle">Pexpect allows easy control of interactive console applications.</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.pythonware.com/products/pil/">PIL</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.pythonware.com/products/pil/license.htm">MIT</a></td>
<td valign="middle">Image processing library</td>
<td class="center-txt">
Image
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/plotly">plotly</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Python plotting library for collaborative, interactive, publication-quality graphs.</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.dabeaz.com/ply/">ply</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.dabeaz.com/ply/">BSD</a></td>
<td valign="middle">Python implementation of lex and yacc</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://olgabot.github.io/prettyplotlib">prettyplotlib</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/olgabot/prettyplotlib/blob/master/LICENSE.txt">MIT</a></td>
<td valign="middle">Painlessly create beautiful default matplotlib plots.</td>
<td class="center-txt">
Plot
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://code.google.com/p/psutil/">psutil</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">OS interface to processes and tasks</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://pypi.python.org/pypi/py">py</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Library with cross-python path, ini-parsing, io, code, log facilities</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://rhodesmill.org/pyephem/">pyephem</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.gnu.org/licenses/lgpl.txt">LGPL</a></td>
<td valign="middle">Scientific-grade astronomy routines</td>
<td class="center-txt">
Astronomy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pyflakes">pyflakes</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Static analysis of Python code to detect common errors</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pygments.org/">Pygments</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://bitbucket.org/birkenfeld/pygments-main/src/fbeebf94f4d2/LICENSE">BSD</a></td>
<td valign="middle">Code syntax highlighting package written in Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/pyOpenSSL">pyOpenSSL</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://bazaar.launchpad.net/~exarkun/pyopenssl/trunk/view/head:/LICENSE">Apache License 2</a></td>
<td valign="middle">Python bindings to the OpenSSL library</td>
<td class="center-txt">
Network
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/PySal">pysal</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A library of spatial analysis functions.</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.pytables.org/">PyTables</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.pytables.org/moin/License">BSD</a></td>
<td valign="middle">Hierarchical datasets for extremely large data</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pytest.org/">pytest</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Simple powerful testing with Python</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://pyyaml.org/wiki/PyYAML">PyYAML</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">YAML parser and emitter</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://github.com/ellisonbg/pyzmq">pyzmq</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/zeromq/pyzmq/blob/master/COPYING.BSD">BSD-like</a></td>
<td valign="middle">Binding to the ZeroMQ fast messaging library</td>
<td class="center-txt">
Middleware
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://docs.python-requests.org/en/latest/">requests</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a></td>
<td valign="middle">Elegant and simple HTTP library for Python</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://scikit-learn.sourceforge.net/">scikit_learn</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Machine learning and data mining routines</td>
<td class="center-txt">
ML
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.scipy.org/">SciPy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/scipy/scipy/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Libraries for mathematics, science, and engineering</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="http://www.scons.org">SCons</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">Pythonic substitute for Make</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://stanford.edu/~mwaskom/software/seaborn/">seaborn</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Seaborn: statistical data visualization</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://packages.python.org/setuptools/">setuptools</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.python.org/psf/license/">PSF</a></td>
<td valign="middle">Easily download, build, install, upgrade, and uninstall Python packages</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr>
<td valign="middle">
<a target="_blank" href="https://github.com/Toblerity/Shapely">Shapely</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/Toblerity/Shapely/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Geometric objects, predicates, and operations</td>
<td class="center-txt">
Geo
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://cheeseshop.python.org/pypi/simplegeneric">simplegeneric</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://cheeseshop.python.org/pypi/simplegeneric">ZPL 2.1</a></td>
<td valign="middle">Simple generic functions (similar to Python's own len(), pickle.dump(), etc.)</td>
<td class="center-txt">
Dev
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://sphinx-doc.org/">Sphinx</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Creates intelligent and beautiful project documentation</td>
<td class="center-txt">
Doc
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.SQLAlchemy.org/">SQLAlchemy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/mit-license.php">MIT</a></td>
<td valign="middle">SQL toolkit and Object Relational Mapper</td>
<td class="center-txt">
Data
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://statsmodels.sourceforge.net/">statsmodels</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/statsmodels/statsmodels/blob/master/LICENSE.txt">BSD</a></td>
<td valign="middle">Statistical computations and models for use with SciPy</td>
<td class="center-txt">
Scipy
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://code.google.com/p/sympy/">SymPy</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Symbolic mathematics library</td>
<td class="center-txt">
Math
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.tornadoweb.org/">Tornado</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a></td>
<td valign="middle">Scalable, non-blocking web server</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://virtualenv.pypa.io/">virtualenv</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://virtualenv.pypa.io/">MIT</a></td>
<td valign="middle">Virtual Python Environment builder</td>
<td class="center-txt">
Sys
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://pypi.python.org/pypi/werkzeug">werkzeug</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">Advanced WSGI utility modules</td>
<td class="center-txt">
Web
</td>
</tr>
<tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="https://github.com/pydata/xarray">xarray</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/pydata/xarray">Apache</a></td>
<td valign="middle">N-D labeled arrays and datasets in Python</td>
<td class="center-txt">
Math
</td>
</tr><tr class="onlyCanopyFree">
<td valign="middle">
<a target="_blank" href="http://www.python-excel.org/">xlrd</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="https://github.com/python-excel/xlrd/blob/master/xlrd/licences.py">BSD</a></td>
<td valign="middle">Extract data from Microsoft Excel (tm) spreadsheet files</td>
<td class="center-txt">
Excel
</td>
</tr><tr>
<td valign="middle">
<a target="_blank" href="http://xlsxwriter.readthedocs.org/">xlsxwriter</a>
</td>
<td valign="middle" class="center-txt heavy"><a target="_blank" href="http://www.opensource.org/licenses/bsd-license.php">BSD</a></td>
<td valign="middle">A Python module for creating Excel XLSX files.</td>
<td class="center-txt">
Excel
</td>
</tr>
</tbody>
<tfoot></tfoot>
</table>

1844
userguide/zh/notebook/r.md Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,10 @@
## 页面介绍 ##
Docklet的主要操作页面:
* [面板](dashboard.md)
* [配置](config.md)
* [状态](status.md)
* [主机](hosts.md)
* [用户](users.md)
* [管理](admin.md)

View File

@ -0,0 +1,4 @@
## 管理 ##
管理(Admin)页面只由具有 *管理员(admin)* 角色的用户访问。

View File

@ -0,0 +1,20 @@
## 配置 ##
配置(Config)是对工作区进行配置操作的界面,
主要是对虚拟集群的节点进行配置,以及对自己的镜像进行管理。
下图显示了 *tee* 工作区的配置页面:
<img src="../images/config-workspace.png" width="600" alt="workspace
config">
在这里,可以为工作区增加或删除节点。上图显示 *tee* 工作区现在有两个节点,
并显示每个节点的名称、IP地址、状态、镜像信息等。
可以点击 **Save** 保存工作区,将其做成一个镜像,名为 *myimage*,方便下次使用。
此时的镜像列表信息如下图所示:
<img src="../images/config-image.png" width="700" alt="image config">
可以对保存的镜像进行管理,将其**共享(Share)**出去,造福他人,
免去他们繁琐的配置步骤。

View File

@ -0,0 +1,49 @@
## 面板 ##
面板(Dashboard)是工作区(Workspace)的操作界面,
在这里用户可以看到其拥有的所有workspace的状态
进行工作区的创建,启动,停止,删除操作,并可以进入工作区。
初始时工作区列表为空,用户需要创建一个工作区。
点击 **Add Workspace**,进入创建工作区的页面,选择镜像,创建工作区
**tee**,如下图所示:
<img src="../images/workspace-create.png" width="600" alt="add
workspace">
创建成功后,工作区列表如下图所示:
<img src="../images/dashboard-stop.png" width="600" alt="dashboard
stopped status">
图中显示已经创建了一个名为 *tee* 的工作区,其状态为 **Stopped**
此时可以进行 **启动(Start)****删除(Delete)** 工作区的操作。
被停止的 工作区 不会被销毁,下次仍然可以继续使用它。
推荐在不运行作业的时候停止 工作区 以减少服务器负载。
点击 **Start**,启动工作区,工作区状态改变,如下图所示:
<img src="../images/dashboard-start.png" width="600" alt="dashboard
running status">
此时显示其状态为 **Running**,可进行 **停止(Stop)****进入(Go)**
工作区的操作。也可以点击工作区的名字 *tee*
可以进入工作区的 [配置](config.md) 页面;点击状态**Runnig**
进入工作区的 [状态](status.md) 页面。
**注意**:
1. 如果在刚创建之后马上点击 **Go** ,可能会出现`503: Proxy Target
Missing` 错误,这很可能是因为后台尚未初始化完成。
稍后再点击 **Go**,就会正常进入工作区。
1. 如果用户点击 **Go** 时出现
```
Not Found
The requested URL was not found on the server. If you entered the URL
manually please check your spelling and try again.
```
可能是因为系统因为某种原因重启用户需要自己重新启动Workspace然后再进入。

View File

@ -0,0 +1,22 @@
## 主机 ##
主机(Hosts)页面仅由具有 *管理员(admin)* 角色的用户访问。
主机页面显示了Docklet物理集群中各主机的状态和配置信息
包括IP地址、状态、运行的容器节点、所消耗的资源等。
下图展示了一个测试环境中的Docklet物理集群信息:
<img src="../images/hosts.png" width="600" alt="hosts">
点击节点的 **Realtime** 按钮,可以看到该主机的配置信息:
<img src="../images/hosts-summary-config.png" width="600" alt="
summary config of one host">
也可以看到该主机的实时资源消耗:
<img src="../images/hosts-summary-perf.png" width="600" alt="
realtime performance of one host">

View File

@ -0,0 +1,12 @@
### 状态 ###
状态(Status) 是虚拟集群的监控界面。在这里,可以看到工作区所在集群的负载情况,
包括CPU和Memory的使用情况。
下图展示了工作区 *tee* 所在虚拟集群的状态概览:
<img src="../images/status.png" width="600" alt="vcluster status">
点击 **Realtime** 可看到节点的CPU和Memory的实时使用情况如下图所示:
<img src="../images/status-summary.png" width="600" alt="vcluster status summary of one node">

View File

@ -0,0 +1,22 @@
## 用户 ##
用户(Users)页面仅由具有 **管理员(admin)** 权限的用户访问。
该页面提供对docklet中用户信息展示、修改功能。
同时可以添加用户、用户组。
### 注册与激活 ###
目前docklet不支持自由注册可以使用外部的认证系统登录如pam
第一次使用外部账号认证登录时,账号处于未激活状态,登录之后页面左上角会显示一条待激活的通知,未激活的账号不能使用一些特定功能。
<img src="../images/user_init.png" width="300" alt="add workspace">
点击该通知之后会进入激活页面需要在该页面中填写E-mail、学号/职工号、部门、真实姓名、注册原因提交申请之后管理员会在3天之内审核该申请。
提交申请表单之后浏览器会自动登出需要重新登录刷新用户信息。在审核期间docklet页面的左上角会显示等待审核的通知点击即可关闭。
## 试用 ##
docklet支持用户以访客模式试用在你的帐号审核通过之前你可以通过点击主页的have a try按钮进入
试用界面在试用模式下你可以点击Go按钮进入到一个workspace中。

View File

@ -0,0 +1,8 @@
# 在终端中开发 #
* [安装的软件包列表](list.md)
* [C/C++示例](c.md)
* [Java示例](java.md)
* [MPI示例](mpi.md)
* [Spark示例](spark.md)
* [LaTeX示例](latex.md)

Some files were not shown because too many files have changed in this diff Show More