Merge remote-tracking branch 'remotes/origin/staging' into shapeways_printing_updated

Conflicts:
	blockly/src/turtle/turtle.js
	dashboard/public/blockly

Author: bcjordan

.gitignore

@@ -4,9 +4,10 @@
 .dropbox
 .idea/
 /.gdrive_session
+/chef/.chef/
+/crontab
+/dashboard/public/blockly
 /.dropbox.cache
-/aws/secrets.key
-/aws/secrets.tgz
 /aws/secrets/
 /aws/solr/
 /build-started
@@ -16,4 +17,7 @@
 \#*#
 /i18n/code.org/crowdin.yaml
 /i18n/hourofcode.com/crowdin.yaml
-/i18n/curriculum/crowdin.yaml
+/i18n/curriculum/crowdin.yaml
+/dashboard/config/newrelic.yml
+/pegasus/config/newrelic.yml
+

README.md

@@ -14,11 +14,11 @@ Many Windows developers have found that setting up an Ubuntu virtual machine is
 
 ## Install OS-specific prerequisites
 
-### OS X Mavericks
+### OS X Mavericks / Yosemite
 
-1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"`
-1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build`
-  1. If it complains about an old version of `X`, run `brew unlink X` and run `brew install X` again
+1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`
+1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build coreutils`
+  1. If it complains about an old version of `<package>`, run `brew unlink <package>` and run `brew install <package>` again
 1. Set up MySQL
   1. Have launchd start mysql at login: `ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents`
   1. Start mysql now: `launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist`
@@ -31,32 +31,23 @@ Many Windows developers have found that setting up an Ubuntu virtual machine is
 
 ### Ubuntu 14.04
 
-1. `sudo apt-get install -y aptitude`
 1. `sudo aptitude update`
 1. `sudo aptitude upgrade`
-1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl`
+1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs npm openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl pdftk`
   * **Hit enter and select default options for any configuration popups**
-1. `sudo aptitude install npm`
-1. `sudo ln -s /usr/bin/nodejs /usr/bin/node`
-1. `sudo npm update -g npm`
-1. `sudo npm install -g grunt-cli`
 
 ## Common setup
 
 1. `git clone https://github.com/code-dot-org/code-dot-org.git`
 1. `gem install bundler`
+1. `rbenv rehash` (if using rbenv)
 1. `cd code-dot-org/aws`
 1. `bundle install`
-1. `cd ../dashboard`
-1. `bundle install`
-1. `bundle exec rake db:create db:schema:load seed:all`
-1. `cd ../pegasus`
-1. `bundle install`
-1. `echo CREATE DATABASE pegasus_development | mysql -uroot`
-1. `rake db:migrate`
-1. `rake seed:migrate`
+1. `cd ..`
+1. `rake install`
 
 ## Organizational Structure
+
 Our code is segmented into four parts:
 
 * Blockly Core is the visual programming language platform used for the interactive tutorials.
@@ -70,36 +61,57 @@ Our code is segmented into four parts:
   * [Teacher Dashboard](http://code.org/teacher-dashboard)
 
 ## Running Dashboard
-1. `cd code-dot-org/dashboard`
-2. `bundle exec rails server`
-3.  Note: after major code updates (or if something seems broken), run `bundle exec rake db:migrate seed:all`
-4.  Visit [http://localhost.studio.code.org:3000/](http://localhost.studio.code.org:3000/)
+
+1. `cd code-dot-org`
+2. `rake build:dashboard` (Generally, do this after each pull)
+3. `bin/dashboard-server`
+4. Visit [http://localhost.studio.code.org:3000/](http://localhost.studio.code.org:3000/)
 
 ## Running Pegasus
 
-1. `cd code-dot-org/pegasus`
-2. `./up`
-3. Note: after major code updates (or if something seems broken), run `rake db:migrate seed:migrate`
+1. `cd code-dot-org`
+2. `rake build:pegasus` (Generally, do this after each pull)
+3. `bin/pegasus-server`
 4. Visit [http://localhost.code.org:9393/](http://localhost.code.org:9393/)
 
 ## Building Blockly and Blockly-core (optional)
 
-The learn.code.org default dashboard install includes a static build of blockly, but if you want to make modifications to blockly or blockly-core:
+The learn.code.org default dashboard install includes a static build of blockly, but if you want to make modifications to blockly or blockly-core you'll want to enable building them in the build:
+
+### Enabling Blockly Builds
+
+You'll need to do this once:
+
+1. OS X:
+  1. Install the [Java 8 JDK](http://www.oracle.com/technetwork/java/javase/downloads/index.html)
+  1. Install [XQuartz](http://xquartz.macosforge.org/trac) (NOTE: This is required to build the Canvas dependency).
+1. `cd code-dot-org`
+1. Edit `locals.yml`
+  1. Add `build_blockly: true`
+  1. Add `build_blockly_core: true`
+  1. Add `use_my_blockly: true`
+1. `rake install`
 
-1. `cd code-dot-org/dashboard`
-1. `bundle exec rake 'blockly:dev[../blockly]'`
-  * This symlinks to dashboard reference the dev version of blockly
-1. Follow the blockly build instructions at `blockly/README` or blockly-core build instructions at `blockly-core/README`
+This configures your system to build blockly (and blockly-core) whenever you run `rake build` and to use the version of blockly that you build yourself.
+
+### Building Blockly and Blockly-Core
+
+1. `cd code-dot-org`
+1. `rake build`
+
+This will build everything you have set to build in `locals.yml`.
+
+You can use `rake build:blockly` and `rake build:blockly_core` to build a specific project.
+
+You can also set `build_dashboard: false` and/or `build_pegasus: false` in `locals.yml` if you don't need to build these frequently. They default to `true`.
 
 ## Contributing
 
 We'd love to have you join our group of contributors!
 
 ### Before You Push
 
-Anyone who would like to contribute to **[code.org](https://github.com/code-dot-org/)** projects **must read and sign the Contribution License Agreement**. We aren't able to accept any pull requests from contributors who haven't signed the CLA first.
-
-For the time being—email [[email protected]](mailto:[email protected]) to get an electronic CLA to sign (takes less than a minute).
+Anyone who would like to contribute to **[code.org](https://github.com/code-dot-org/)** projects **must read and sign the [Contributor License Agreement](https://na2.docusign.net/Member/PowerFormSigning.aspx?PowerFormId=8eb90665-c9f7-4b06-81a5-11d12020f251)**. We can't accept pull requests from contributors who haven't yet signed the CLA.
 
 ### Getting Started Contributing
 
@@ -145,7 +157,7 @@ Contributors should follow the GitHub [fork-and-pull model](https://help.github.
     - `git push origin branch_name`
 3. Go to the code-dot-org GitHub page
     - [https://github.com/code-dot-org/code-dot-org](https://github.com/code-dot-org/code-dot-org)
-4. For your submissinon to be reviewed
+4. For your submission to be reviewed
     - Click on the "Pull Request" link, look over and confirm your diff
     - Submit a pull request for your branch to be merged into staging
     - For bonus points, include screenshots in the description. Command + Ctrl + Shift + 4 in OS X lets you copy a screen selection to your clipboard, which GitHub will let you paste right into the description

Rakefile

@@ -1,7 +1,20 @@
 require_relative './deployment'
+require 'os'
 require 'cdo/hip_chat'
 require 'cdo/rake_utils'
 
+def create_pegasus_db()
+  db = URI.parse CDO.pegasus_db_writer
+  command = [
+    'mysql',
+    "--user=#{db.user}",
+    "--host=#{db.host}",
+  ]
+  command << "--execute=\"CREATE DATABASE IF NOT EXISTS #{CDO.pegasus_db_name}\""
+  command << "--password=#{db.password}" unless db.password.nil?
+  system command.join(' ')
+end
+
 def with_retries(count=5)
   begin
     yield if block_given?
@@ -23,19 +36,16 @@ end
 namespace :build do
 
   task :configure do
-    if CDO.chef_managed && !(rack_env?(:production) && CDO.name =='daemon')
+    if CDO.chef_managed && !(rack_env?(:production) && CDO.name =='daemon') && !rack_env?(:test) && !rack_env?(:staging)
       HipChat.log 'Applying <b>chef</b> profile...'
       RakeUtils.sudo 'chef-client'
     end
 
-    Dir.chdir(aws_dir) do
-      unless CDO.chef_managed
+    unless CDO.chef_managed
+      Dir.chdir(aws_dir) do
         HipChat.log 'Installing <b>aws</b> bundle...'
         RakeUtils.bundle_install 
       end
-
-      HipChat.log 'Configuring <b>aws</b>...'
-      RakeUtils.rake
     end
   end
 
@@ -81,8 +91,6 @@ namespace :build do
       RakeUtils.bundle_install
 
       if CDO.daemon || rack_env?(:levelbuilder)
-        RakeUtils.rake 'db:create' unless rack_env?(:production)
-
         HipChat.log 'Migrating <b>dashboard</b> database...'
         RakeUtils.rake 'db:migrate'
 
@@ -98,8 +106,9 @@ namespace :build do
       HipChat.log 'Starting <b>dashboard</b>.'
       RakeUtils.start_service CDO.dashboard_unicorn_name unless rack_env?(:development)
 
-      # report deployment to honeybadger
-      RakeUtils.system 'rake', "honeybadger:deploy TO=#{rack_env} REVISION=`git rev-parse HEAD`"
+      if rack_env?(:production)
+        RakeUtils.system 'rake', "honeybadger:deploy TO=#{rack_env} REVISION=`git rev-parse HEAD`"
+      end
     end
   end
 
@@ -163,44 +172,50 @@ task :build => ['build:all']
 ##################################################################################################
 
 namespace :install do
+  
+  task :blockly do
+    if rack_env?(:development) && !CDO.chef_managed
+      Dir.chdir(blockly_dir) do
+        blockly_build = CDO.use_my_blockly ? blockly_dir('build/package') : 'blockly-package'
+        RakeUtils.ln_s blockly_build, dashboard_dir('public','blockly')
+      end
+
+      if OS.linux?
+        RakeUtils.sudo_ln_s '/usr/bin/nodejs', '/usr/bin/node'
+        RakeUtils.sudo 'npm', 'update', '-g', 'npm'
+        RakeUtils.sudo 'npm', 'install', '-g', 'grunt-cli'
+      elsif OS.mac?
+        RakeUtils.system 'brew install node'
+        RakeUtils.system 'npm', 'update', '-g', 'npm'
+        RakeUtils.system 'npm', 'install', '-g', 'grunt-cli'
+      end
+    end
+  end
 
   task :dashboard do
-    unless CDO.chef_managed
-      Dir.chdir(aws_dir) { RakeUtils.rake }
+    if rack_env?(:development) && !CDO.chef_managed
       Dir.chdir(dashboard_dir) do
         RakeUtils.bundle_install
-        unless rack_env?(:production)
-          RakeUtils.rake 'db:create'
-          RakeUtils.rake 'db:schema:load'
-        end
-        RakeUtils.sudo_ln_s dashboard_dir('config/init.d'), File.join('/etc/init.d', CDO.dashboard_unicorn_name)
-        RakeUtils.sudo 'update-rc.d', CDO.dashboard_unicorn_name, 'defaults'
-        RakeUtils.sudo_ln_s dashboard_dir('config/logrotate'), File.join('/etc/logrotate.d', CDO.dashboard_unicorn_name)
-        RakeUtils.sudo 'service', CDO.dashboard_unicorn_name, 'start'
+        RakeUtils.rake 'db:create'
+        RakeUtils.rake 'db:schema:load'
       end
     end
   end
 
   task :pegasus do
-    unless CDO.chef_managed
-      Dir.chdir(aws_dir) { RakeUtils.rake }
+    if rack_env?(:development) && !CDO.chef_managed
       Dir.chdir(pegasus_dir) do
         RakeUtils.bundle_install
-        unless rack_env?(:production)
-          RakeUtils.system 'echo', "'CREATE DATABASE IF NOT EXISTS #{CDO.pegasus_db_name}'", '|', 'mysql', '-uroot'
-          RakeUtils.rake 'db:migrate'
-          RakeUtils.rake 'seed:migrate'
-        end
-        RakeUtils.sudo_ln_s dashboard_dir('config/init.d'), File.join('/etc/init.d', CDO.pegasus_unicorn_name)
-        RakeUtils.sudo 'update-rc.d', CDO.pegasus_unicorn_name, 'defaults'
-        RakeUtils.sudo 'service', CDO.pegasus_unicorn_name, 'start'
+        create_pegasus_db
+        RakeUtils.rake 'db:migrate'
+        RakeUtils.rake 'seed:migrate'
       end
     end
   end
 
   tasks = []
   #tasks << :blockly_core if CDO.build_blockly_core
-  #tasks << :blockly if CDO.build_blockly
+  tasks << :blockly if CDO.build_blockly
   tasks << :dashboard if CDO.build_dashboard
   tasks << :pegasus if CDO.build_pegasus
   task :all => tasks

aws/.gitignore

@@ -12,15 +12,3 @@
 /.website-ci-built
 /.websites-built
 /.websites-test-built
-/bashrc
-/crontab
-/crontab.installed
-/dashboard_unicorn.rb.pid
-/pegasus_unicorn.rb.pid
-/postfix/main.cf
-/postfix/sasl_password
-/postfix/sasl_password.db
-/profile
-/varnish.config
-/varnish.secret
-/varnish.vcl