underfitting and overfitting , n and p

Overfitting refers to a model that models the training data too well.

Underfitting refers to a model that can neither model the training data nor generalize to new data.

We have p parameters and n sample.
over fitting results from trying to estimate too many parameters from too small a sample, when p>n

if we remove one feature, we will decrease the degree of overfitting .

ECS/EKS container services , docker, airflow, snowflake database

ECS/EKS container services

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

Docker is a software platform for building applications based on containers — small and lightweight execution environments that make shared use of the operating system kernel but otherwise run in isolation from one another. While containers as a concept have been around for some time, Docker, an open source project launched in 2013, helped popularize the technology, and has helped drive the trend towards containerization and microservices in software development that has come to be known as cloud-native development.

Docker is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime. Using Docker, you can quickly deploy and scale applications into any environment and know your code will run.

containers amazon offers

https://aws.amazon.com/containers/services/

I used EMR before

https://aws.amazon.com/emr/


a tutorial for docker

https://www.youtube.com/watch?v=K6WER0oI-qs



airflow: Airflow is a platform to programmatically author, schedule and monitor workflows.

a short summary

https://blog.insightdatascience.com/airflow-101-start-automating-your-batch-workflows-with-ease-8e7d35387f94


https://airflow.apache.org/project.html

how to install

https://airflow.apache.org/installation.html

video tutorial

https://www.youtube.com/watch?v=AHMm1wfGuHE






snowflake database: cloud based data warehouse

https://docs.snowflake.net/manuals/user-guide/getting-started-tutorial.html

7 tips to learn programming faster

https://www.codingdojo.com/blog/7-tips-learn-programming-faster

#3 will land you a job

1. learn by doing
2. grasps the fundamentals for long-term benefit
3.code by hand, using a pen and write on paper
4.ask for help
5.seek out more online resources
6. don't just read the sample code, tinker with it
7. take breaks when debugging

How to run a python script on atom

how to run a python script on atom :

mac  shift + command + I
mac command +I

linux/windows : SHIFT + Ctrl + B

A thesis from a Phd and what he has done since graduation

Here is a thesis from a Phd

https://lib.dr.iastate.edu/etd/13537/

The title of the thesis is

A balanced approach to the multi-class imbalance problem


And after graduation, the author did not work for companies, he opens his consulting firm instead

Omni Analytics Group

https://omnianalytics.io/

one good sql tutorial and some good machine learning channels

one good sql tutorial

https://www.youtube.com/watch?v=nWeW3sCmD2k


some good machine learning channels

https://www.youtube.com/user/joshstarmer/videos


https://www.youtube.com/user/mathtutordvd/videos

https://www.youtube.com/channel/UCq8JbYayUHvKvjimPV0TCqQ/videos

https://www.youtube.com/user/edurekaIN/videos

https://www.youtube.com/channel/UC8butISFwT-Wl7EV0hUK0BQ/videos