How I connect to Amazon Redshift through Aginity

To connect to Amazon Redshift through Aginity, you need to have Red Shift Native Driver on Aginity.

At beginning, I thought I could change my the other Driver in Aginity to Red Shift Native. But that's not the case.

You need to download and install a separate Amazon Redshift Aginity Workbench:


https://www.aginity.com/redshift-download-link/

And you follow the instructions to get there.

And you will see it on windows:

Here is a useful video:

https://www.aginity.com/workbench/workbench-redshift-basics/

Connect to SQLworkbench from Amazon Redshift

Here is how on amazon.com


http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-using-workbench.html


Here is how on youtube:

https://www.youtube.com/watch?v=ixcL3AMqZik

Generally a python class structure

When we study a python class structure, we will have __init__, self, etc. But in general, when we write a python class, it is usually like:

class Solution:
       def  topR(self, num,b)


To run a class, it is usually like:

aa=Solution()

kk=aa.topR(x1,x2)

And the self can not be omitted. Otherwise, we will get an error.

Here is an example, top k frequentest number in a list:

import collections
import operator
class Solution:
    def topN(self, num,k):
         b=collections.defaultdict(int)

         for i in num:
             b[i]+=1

         sorted_x = sorted(b.items(), key=operator.itemgetter(1),reverse=True)

         return sorted_x[:k]

aa=Solution()
print aa.topN([1,1,1,2,3,3,3,5,5,5,5,9,9,9,9,9,9,9,9],2)

Results:

[(9, 8), (5, 4)]

Another python class example

class Shape:
    def __init__(self,x1,y1):
          self.x=x1
          self.y=y1
          self.description="This shape has not been described yet"
          self.author="Nobody has claimed to make this shape yet"

     def area(self):
           return self.x*self.y

    def perimeter(self):
          return 2*self.x+2*self.y

    def describe(self,text):
          self.description=text

    def authorName(self,text):
          self.author=text

    def scaleSize(self,scale):
         self.x=self.x*scale
         self.y=self.y*scale


to run:

aa=Shape(5,6)

aa.scaleSize(0.2)

print aa.scaleSize(0.2)

output:

(0.2, 0.24000)

Another Python class example

class Rocket():
    def __init__(self):
            self.x=0
            self.y=0
    def move_up(self):
        self.y+=1
        return self.y


my_rockets=[]

for x in range(0,5):
        new_rocket=Rocket()
        my_rockets.append(new_rocket.move_up())

for rocket in my_rockets:
        print(rocket)


output:

1
1
1
1
1


If you change a little:

class Rocket():
    def __init__(self):
            self.x=0
            self.y=0
    def move_up(self):
        self.y+=1
        return self.y


my_rockets=[]

for x in range(0,5):
        new_rocket=Rocket()
        my_rockets.append(new_rocket)

for rocket in my_rockets:
        print(rocket)


output will be:

<_main_.Rocket instance at ox7f7fc3fa5878>
<_main_.Rocket instance at ox7f7fc3fa58c0>

...

each time, you got a new rocket.

Another Python class example

class Point:
     def __init__(self, a,b):
          self.x=a
          self.y=b
    def  distance_from_origin(self):
         return ((self.x)**2+(self.y)**2)**0.5


d=Point(3,4)

print d.distance_from_origin()

output:

5.0

Another class example

Another class example:

class Employee:
         empCount=0
     
         def __init__(self,name,salary):
                  self.nam=name
                  self.sala=salary
                  Employee.empCount+=1

         def displayCount(self):
                 print "Total Employee %d"  % Employee.empCount

         def  DisplayEmployee(self):
               print "Name:", self.nam,",Salary:",self.sala


c=Employee("Elva",200)

c.displayCount()
c.DisplayEmployee()


output:

Total Employee 1

Name: Elva, Salary: 200


c=Employee("Kate",300)
c.displayCount()
c.DisplayEmployee()

output:

Total Employee 2

Name: Kate, Salary: 300

d=Employee("Dojo",500)
d.displayCount()
d.DisplayEmployee()

output:

Total Employee 3
Name: Dojo, Salary: 500

Python classes

Python classes:

http://introtopython.org/classes.html

Python object, instance, classes etc:

https://www.tutorialspoint.com/python/python_classes_objects.htm


Python instance and object illustration with pics:

An example looking closer to class in python

An example looking closer to class in python:

class Complex:
       def  __init__(self,realpart,imagpart):
             self.r=realpart
             self.i=imagpart

x=Complex(3.0,-4.5)
print x.r, x.i

You will get the output:

3.0  -4.5

Python classes, objects, instances doc

Python classes, objects, instances doc

https://docs.python.org/2/tutorial/classes.html

VNC, tele desktop

VNC, tele desktop, connect windows with linux, a virtual machine.

http://www.tightvnc.com/release-2.7.10.php

A good python book

A Primer on Scientific Programming with Python

Amazon link: 


https://www.amazon.com/Scientific-Programming-Computational-Science-Engineering/dp/3642302920

How to switch to another disk and how to get ipconfig work on command prompt

How to switch to another disk and how to get ipconfig work on command prompt.

If you want to switch from C drive to G drive, you can try:

cd  /d G:\

If you want to know your IP, you can use ipconfig. Sometimes it won't work. Try to go to windows\system32, and then type ipconfig.

C:\Windows\System32\ipconfig

How to run a python class, and ow to run R script on linux

Here is a python class:



    class Student:
         def   __init__(self, name, age):
                 self.name=name
                 self.age=age
         def   say_hello(self):
                print(self.name, "say hello")

And, to run it, type the following in the same program:

alice=Student("Alice", 16)

alice.say_hello()

Do not forget the parenthesis.

Save the program as python1.py

And on linux run:

python python1.py

And you will see on the screen:

('Alice', 'say hello')

Here is another example:

class Base1:

def __init__(self,x,y):

self.x=x

self.y=y

def add1(self):

z=self.x+self.y

print(z)

def main():

foo=Base1(1,2)

foo.add1()

if __name__=="__main__":

main()

And you will see 3 was printed on screen.

To run an R program on linux, write an R program and save as rprog1.R

a=c(1,2,3)

for ( i in 1:3)

{ print(i)}

And run Rscript rprog1.R on linux, you will see:

[1] 1

[2] 2

[3] 3

Text mining websites and books etc

text mining with R:

http://tidytextmining.com/

text mining website:

http://abbottanalytics.blogspot.com/search/label/text%20mining


http://text-analytics101.rxnlp.com/

Delete the duplicated rows in a single line without creating new table.

Delete the duplicated rows in a single line without creating new table. 

ALTER IGNORE TABLE table1   ADD UNIQUE INDEX (name1,gender);

Python class self

wiki:

https://en.wikipedia.org/wiki/This_(computer_programming)

quora:

https://www.quora.com/What-does-self-mean-in-python-class-Why-do-we-need-it







A class have methods. Also, a class can have multiple objects. Right? Now when any object calls a method of the class, how would the method know which object has called it?
For example, if any of your family member calls your name, how do you recognize that its your mother or father or brother, if you are sitting in a different room? You recognize the voice. Right?

Similarly, self represents the voice of the object.

Example -

class myClass:
    def __init__(self):
        self.x = []
        self.x.append(1)

    def print_x(self):
        print (self.x)

obj1 = myClass()
obj1.print_x()    # prints [1]

obj2 = myClass()
obj2.print_x()    # prints [1],  not "[ 1, 1]"


Now see, both the times [1] is printed even when we used "append".

Whenever you write - obj1.print_x(), self passes the instance of "obj1" . Now "print_x()" would know that "obj1" has called it. Therefore, obj1 would have its own copy of "x" not matter how many times method "print_x" is called by other objects. Same would be the case with obj2.

In other object oriented languages too, when you call a method using an object, instance of an object get passed automatically(like "this" pointer in C++), but we don't have to write that explicitly. Its just that in python you have to be explicit :)

Websites to get programming, data science certificates and data science camp

Websites to get programming, data science certificates:

freecodecamp

udemy

teamtreehouse

coursera

edx

udacity.com

dataquest

datacamp

o'reilly

springboard

data origami

youtube 

data science camp:

insightdatascience

thedataincubator

datacamp

Collections of complex SQL queries I

Collections of complex SQL queries I

we have the table:

id name department 'id of manager', doj (date of joining)

insert into employee values(1,'John','IT',9,'05-08-2010');

1. list the employees who are not managers

2. manager with only one reportee

3. what is the month with most hiring?

4.what is the experience gap between the first employee and the latest?

5. name the manager with most reportees?

6.list managers who joined after the reportees

7.department with most managers and how many?

8.Delete duplicate rows using a single statement and no table creation.
9.Identify the second highest salary from emp table having salary related data.
10.Statement to determine the oracle version used .
11. Query to accept a date and return the fist date of the quarter in which the date exists.
12.Table A has 100 rows, Table B has Zero rows so number of rows returned from  below query
select a.* from a, b;
13. Set of statements executed in the below sequence.
select count(*) from a;

Count(*)
-------
100

Then 1 row inserted in table a.

rollback;
again 3 rows inserted in the table a;
select count(*) from a;
count(*)
--------
103

create table b using select * from a;

rollback;

so what would be the o/p of below select statement
select count(*) from a; 
?
14 .Statement to print numbers from 1 to 100 in sequence.
15.Create a table with no rows structure similar to existing table using an SQL statement.
16. Few queries related to analytical functions. lead, lag, rank, dense_rank
17. Table Employees

NAME  GENDER
----  ------ 
A    MALE
B    FEMALE
BB   FEMALE
CC   FEMALE
CD   MALE
DD   MALE
DE   FEMALE


Write a query to give the o/p like below

MALE  FEMALE
----  ------
3     4

Text mining 101

Text mining 101, from here:

http://www.statsoft.com/Textbook/Text-Mining

Amazon Redshift docs

Amazon redshift is an Amazon database/query language.

Here are some docs for it online:

first step:

http://docs.aws.amazon.com/redshift/latest/gsg/redshift-gsg.pdf


query language manual:

https://docs.aws.amazon.com/redshift/latest/dg/redshift-dg.pdf

management:

http://docs.aws.amazon.com/redshift/latest/mgmt/redshift-mgmt.pdf

Atom atom runner shortcut

https://atom.io/packages/atom-runner

on windows, I used the shortcut, alt-r to run a script. And I got the results of the script on the right.

Try to make the Scala working on my Atom

Scala is a relatively complicated programming language.  We need to use the sbt to make it run. 

I tried to make it work on Atom, to make an IDE for Scala on Atom.

I first go to Atom=>File=>Settings=>Install , search "Ensime", and install it. 

And I ctrl+shift+p , to find ensime: Start. and I got an error " no ensime file found".

So I tried to generate ensime file using sbt etc. 

I went to 

http://ensime.org/build_tools/sbt/

http://www.scala-sbt.org/

And download sbt. 

And I follow the following post, the last one, 

"add a plugins folder and plugins.sbt in %username%.sbt\0.13\"

https://www.reddit.com/r/scala/comments/58ysn9/how_to_use_sbt_ensimeconfigure_in_ensime_plugin/

And on the windows cmd prompt, I went to the sbt directory 

And I use the command "sbt" to go to sbt mode. And under sbt mode, I run ensimeConfig. Something wrong. 

And I tried to change the permission on the Java on my computer. 

And I can not change the permission. It is a computer at work. Will try on my own laptop. 

statistics websites etc

statistics websites:

website for statistics websites:

http://www.kdnuggets.com/websites/index.html

three  statistics websites:

http://www.statsoft.com/

http://www.statsoft.com/Textbook


http://stattrek.com/

https://www.r-statistics.com/

Two detailed java tutorial on youtube

These two java tutorials on youtube are very  detailed:

https://www.youtube.com/watch?v=qgMH6jOOFOE&list=PLS1QulWo1RIbfTjQvTdj8Y6yyq4R7g-Al&index=5

https://www.youtube.com/watch?v=_MYiF7VBOgY&list=PL9ooVrP1hQOFR25JoQW3h3n5pBs77e6KU

How to install emacs on ubuntu (proved on my laptop)




Three lines:




sudo add-apt-repository ppa:ubuntu-elisp/ppasudo apt-get updatesudo apt-get install emacs24 ( or any version of your choice of emacs)

You can learn programming from primer.

You can learn programming from primer. Language primers are some concise tutorials and books to guide you to learn programming.

Java primer

http://cs.bc.edu/~donaldja/102/JavaPrimer.html

http://math.hws.edu/eck/cs124/downloads/javanotes7-linked.pdf

http://wwwusers.di.uniroma1.it/~parisi/Risorse/Java%20Primer.pdf


C primer

http://vc.airvectors.net/tscpp.html

C++ primer and notes

http://euclid.nmu.edu/~mkowalcz/cs201f12/A%20Quick%20C++%20Primer.pdf

http://gravitation.web.ua.pt/msampaio/CourseNotes.pdf

https://www.cs.rochester.edu/~nelson/courses/csc_173/review/CppPrimer.html





Python primer

https://hplgit.github.io/primer.html/doc/pub/half/book.pdf

Apache hadoop is open source, free, and there are top 6 hadoop vendors providing big data platforms.

Apache hadoop is open source, free, and there are other 10  hadoop vendors providing big data platforms.


https://www.dezyre.com/article/top-6-hadoop-vendors-providing-big-data-solutions-in-open-data-platform/93

1) Amazon Elastic MapReduce

2) Cloudera CDH Hadoop Distribution

3) Hortonworks Data Platform (HDP)

4) MapR Hadoop Distribution

5) IBM Open Platform

6) Microsoft Azure's HDInsight -Cloud based Hadoop Distrbution

7) Pivotal Big Data Suite

8) Datameer Professional

9) Datastax Enterprise Analytics

10) Dell- Cloudera Apache Hadoop Solution.

Amazon's big data platform is complete.

https://aws.amazon.com/big-data/

It has a lot of things there, hadoop, spark, s3, ec2, redshift, kinesis etc. 

Apache hadoop, and apache hive etc are free. If you have a linux platform, you can install them on your linux. 

Python's version and which version of python to use.

Python has two versions on python.org downloads:

https://www.python.org/downloads/


3.6.0 and 2.7.13

3.6.0 is the most recent one.  But 90% of the python code in industry is written in 2.7.X. And there are a lot of modules of python in python 2.

On this python primer, page 4, it also said that a lot of python code was written in python 2.


https://hplgit.github.io/primer.html/doc/pub/half/book.pdf

On youtube, currently, most of the python tutorial is about python 3.

For example, I like the tutorial from this channel, their python tutorial is about python 3.

https://www.youtube.com/watch?v=BTzav965P7w&t=2650s