Skip to content

The script WordCount.py in the rdd directory has a couple of issues #8

Open
@tappoz

Description

@tappoz

I am running these scripts on Debian Stretch.

Path issues

I had to import os, then add this:

    projRootPath = os.path.dirname(os.path.realpath(__file__)) + "/../"
    lines = sc.textFile(projRootPath + "in/word_count.text")

Otherwise I am unable to run the script from different points in the file system.

Python version

It does not work on python 2. To make it work I had to use this environment variable:

export PYSPARK_PYTHON=python3

This information could be added to a README file.

I can provide a PR in case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions