Skip to content Skip to sidebar Skip to footer
Showing posts with the label User Defined Functions
Pyspark 2.1: Importing Module With Udf's Breaks Hive Connectivity

Pyspark 2.1: Importing Module With Udf's Breaks Hive Connectivity

Post a Comment
I'm currently working with Spark 2.1 and have a main script that calls a helper module that con… Read more Pyspark 2.1: Importing Module With Udf's Breaks Hive Connectivity
Withcolumn With Udf Yields Attributeerror: 'nonetype' Object Has No Attribute '_jvm'

Withcolumn With Udf Yields Attributeerror: 'nonetype' Object Has No Attribute '_jvm'

Post a Comment
I am trying to replace some values in a spark dataframe by using a UDF, but keep on getting the sam… Read more Withcolumn With Udf Yields Attributeerror: 'nonetype' Object Has No Attribute '_jvm'
Spark Dataframe In Python - Execution Stuck When Using Udfs

Spark Dataframe In Python - Execution Stuck When Using Udfs

Post a Comment
I have a spark job written in Python which is reading data from the CSV files using DataBricks CSV … Read more Spark Dataframe In Python - Execution Stuck When Using Udfs
Error 1121 Importing External Library In Pig Udf In Jython

Error 1121 Importing External Library In Pig Udf In Jython

Post a Comment
I'm having a problem using the python library simplejson in jython to write a Pig UDF. I need b… Read more Error 1121 Importing External Library In Pig Udf In Jython