[5 min IT]PySpark

  • .builder — gives access to Builder API to configure the session
  • .master() — determines where the program will run; "local[*]" sets it to run locally on all cores; "local[1]" to run on one core
  • .appName() — optional method to name the Spark Application
  • .getOrCreate() — gets an existing SparkSession or creates new one if none exists




a world traveler

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Retreeb Staking Details

Resilient Microservice Design With Spring Boot — Circuit Breaker Pattern

LEGO Mindstorms: Gateway to Robotics

Tech Lead fired The Best Front-end Developer in a Little Conversation


Lead Positions

Deep Dive Test Automation: Layers of Test Automation

Easy Code : Web Scrapping :Python Scrapy: Spider Input and automation

Just guess include.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Lisa C. L.

Lisa C. L.

a world traveler

More from Medium


Apache Spark: RDDs, DataFrames, Datasets

Sentiment analysis on streaming Twitter data using Kafka, Spark Structured Streaming & Python (Part…

Installing Spark on Windows 10