sum case when pyspark

https://stackoverflow.com/questions/40762066/sum-of-case-when-in-pyspark

https://stackoverflow.com/questions/49524501/spark-sql-sum-based-on-multiple-cases

pyspark timestamp function, from_utc_timestamp function

http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html

https://stackoverflow.com/questions/45469438/pyspark-creating-timestamp-column

regular expression extract pyspark

https://stackoverflow.com/questions/46410887/pyspark-string-matching-to-create-new-column

regular expression for pyspark

https://stackoverflow.com/questions/45580057/pyspark-filter-dataframe-by-regex-with-string-formatting

pyspark sql case when to pyspark when otherwise

https://www.programcreek.com/python/example/98243/pyspark.sql.functions.when

pyspark user defined function

https://stackoverflow.com/questions/34803855/pyspark-dataframe-udf-on-text-column

pyspark sql functions

https://people.apache.org/~pwendell/spark-nightly/spark-master-docs/latest/api/sql/#when

python tips, intermediate

http://book.pythontips.com/en/latest/

Pyspark SQL example

https://www.codementor.io/jadianes/python-spark-sql-dataframes-du107w74i

Another article about python decorator

https://realpython.com/primer-on-python-decorators/

python advanced exercises

https://python.g-node.org/python-summerschool-2011/_media/materials/advanced_python/exercises_and_solutions.pdf

Python tips

Python tips

http://book.pythontips.com/en/latest/

Python's *args and **kwargs

Python's *args and **kwargs

http://book.pythontips.com/en/latest/args_and_kwargs.html

An article about python cache, pickle and decorator

https://datascience.blog.wzb.eu/2016/08/12/a-tip-for-the-impatient-simple-caching-with-python-pickle-and-decorators/

An understandable article about python decorator

Here is an understandable article about python decorator:


https://www.programiz.com/python-programming/decorator

automatically log into ftp on linux and upload file

https://blog.eduonix.com/shell-scripting/how-to-automate-ftp-transfers-in-linux-shell-scripting/

https://www.linux.com/blog/ftp-file-transfer-automated-bash-script

http://www.columbia.edu/kermit/ftpscripts.html

Sample code for automatically log into sftp , and load a file to sftp

sample code

#!/bin/sh
HOST='HOST'
USER='USER'
PASSWD='PASS'


sshpass -p $PASSWD sftp $USER@$HOST << EOF

put file1.txt

EOF
echo "end ftp"
exit