kholohan / docker-hadoop-spark-workbench Public

forked from big-data-europe/docker-hadoop-spark-workbench

Notifications You must be signed in to change notification settings
Fork 0
Star 0

[EXPERIMENTAL] This repo includes deployment instructions for running HDFS/Spark inside docker containers. Also includes spark-notebook and HDFS FileBrowser.

0 stars 381 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.gitignore		.gitignore
README.md		README.md
README.md.orig		README.md.orig
docker-compose.yml		docker-compose.yml
hadoop.env		hadoop.env
spawn-spark-worker.sh		spawn-spark-worker.sh

Repository files navigation

How to use HDFS/Spark Workbench

To start an HDFS/Spark Workbench:

    network create hadoop
    docker-compose up -d

To scale up spark-workers:

    docker-compose scale spark-worker=3

Docs

Motivation behind the repo and an example usage @BDE2020 Blog

About

[EXPERIMENTAL] This repo includes deployment instructions for running HDFS/Spark inside docker containers. Also includes spark-notebook and HDFS FileBrowser.

Readme