Stanley Chan's Note🧠
Search
Search
Search
Dark mode
Light mode
Explorer
⚙️ Stack
Airflow
Airflow
Airflow Architecture
Airflow Intro
Airflow Operators
Airflow Useful tips
Catchup and backfill
Conditional Branch & Trigger Rule
Connection
CRON
Installing python dependencies to airflow docker container
Setup Airflow in Docker
Setup Airflow locally
Subdag
Task Lifecycle
x-com
Alteryx
Add Recursive Sequence to records
Alteryx
Alteryx auto-insight
capture the highest of each group
Condition
create row sum and column sum
Data manipulation
download
expert exam prep
Find number of business days in a date range
ID per group
Indexing using multirow tool
Join two tables horizontally without common column
macro
multirow tool
Number of unique combination source
Numeric
output
Output tool
parsing a HTML file
parsing a XML file
Pivot
Regex
render tool
Server
Spatial
String
tips
Web Scripting
XML parse
AWS
Containers & ECS & EKS 1
Elastic Kubernetes service (EKS)
Amazon Appflow
Amazon Athena
Amazon Comprehend
Amazon DataZone
Amazon Forecast
Amazon Fraud Detector
Amazon Grafana
Amazon GuardDuty
Amazon Inspector
Amazon Kendra
Amazon Lex
Amazon Macie
Amazon Managed Workflows for Apache Airflow (MWAA)
Amazon MQ
Amazon Polly
Amazon Rekognition
Amazon SageMaker
Amazon Textract
Amazon Transcribe
Amazon Translate
API gateway
Architecture Design
Auto Scaling Group
AWS
AWS Application Discovery Service & Application Migration Service
AWS backup
AWS Batch
AWS budgets
AWS Certificate Manager (ACM)
AWS Cloudwatch
AWS Config
AWS Control Tower
AWS Cost Explorer
AWS DataSync
AWS Direct Connect (DX)
AWS Directory Service
AWS EMR
AWS EventBridge (Cloudwatch events)
AWS Global Accelerator
AWS global infrastructure
AWS Glue
AWS Lambda
AWS Local Zones
AWS Opensearch Service
AWS S3
AWS Secrets Manager
AWS Serverless Application Model
AWS Shield
AWS Simple queue service (SQS)
AWS site-to-site VPN
AWS Snowball
AWS Step functions
AWS Transfer Family
AWS Transit Gateway (TGW)
Border Gateway protocol (BGP)101
CICD in AWS
Cloudformation
Cloudfront
CloudHSM
CloudWatch vs CloudTrail vs Config
Cognito
Containers & ECS & EKS
Database Migration Service (DMS)
DynamoDB
EC2
Elastic File System (EFS)
Elasticache
FSx
Good features for architecture
Identity Access Management (IAM)
Introduction of AWS
IPSec VPN fundamentals
Key Management Service (KMS)
Kinesis
Load balancers
Redshift
Relational Database Service (RDS)
Route 53
Simple Notification Service (SNS)
Storage Gateway
VPC
Web Application Firewall (WAF)
Azure
ADF
Availability Sets vs Proximity Group
Azure Aalystics workloads
Azure Aalystics workloads_StanleysMacMini.lan_Mar-06-202631-2025_Conflict
Azure Databricks Architecture
Azure Entra ID
Azure Storage
Azure Storage - General storage GPv2
Azure Synapse Analytics
Azure Synapse Analytics - data loading
Azure Synapse Analytics_StanleysMacMini.lan_Mar-06-202632-2025_Conflict
Cosmos DB
Cosmos DB_StanleysMacMini.lan_Mar-06-202633-2025_Conflict
DB - Non-Relational DBs
DB - Relational DBs
DB security
Microsoft defender for cloud
Monitoring Azure
Non-Relational DBs in Azure
Query Tool
Databrick
1.1 Databrick intro
1.2 Magic command
1.3 dbutilis
2.1 Delta Lake for Spark
2.2 Relational entities
3.1 Querying files
3.2 Writing to Tables
3.3 Advance Transformation
3.4 Higher Order Functions & SQL UDFs
4.1 Structured Streaming
4.2 Incremental Data Ingestion from files
4.3 Multi-Hop architecture
5.1 Delta Live Table (DLT)
5.2 Change Data Capture
5.3 Jobs
5.4 Databricks SQL (DBSQL)
6.1 Data Objects Privileges
6.2 Unity Catalog (UC)
Others
DBT
_DBT_index
--target-path
Access & Group
Advantage
Analystic engineering
Best practice
contract
custom schema
DBT
DBT Exam Topic Review
deployment
Fundamental command
grants
Macro
Materialization
select selector
Slim CI
small knowledge & trick
source (model for staging)
test
Wrong schema resolve
YAML
Docker
ARGuments & ENVironment variables
Docker
Docker command cheatsheet
Docker compose
Docker Multi-staged build
Docker Network
Docker Volume & Bind mount
Dockerfile
Dockerfile Deployments
dockerignore
RUN vs CMD vs ENTRYPOINT
Git
Check staging area & clean the stage
Git
git branch
git cherry-pick
git common command
git fetch
git merge
git pull
git pull vs git fetch
git push
git rebase
git reset
git revert
git stash
git tag
gitattribute file
Github action
gitignore file
HEAD
Push to remote (general)
remote origin
Special tricks - edit previous commit
Tilde & Caret reference
Linux
bash command
bash evnronment
Bash Shell
bashrc file
dos2unix
Expansion
Kernel & Distro
Linux-Unix
shebang
Symbolic links
System Path and Alias
vim command
wsl
Postgres
Initialize Postgres and PG-admin in Docker
Postgres
Power BI
CALCULATE()
Power BI
Python
API
Class & magic method
Connect to postgres
Create & activate python venv
Create & activate python venv &package
Decrorator
Duck typing
Insert Data Into Remote DB
Project Consideration
Python
Snowflake
Snowflake
Snowflake - Architecture
Snowflake - Clustering
Snowflake - Data Masking
Snowflake - Data Unload
Snowflake - Dynamic table
Snowflake - External table
Snowflake - HASH
Snowflake - LATERAL FLATTEN
Snowflake - Materialized View
Snowflake - Micro-Partition (MPs)
Snowflake - Multi-cluster warehouse
Snowflake - Query Profile
Snowflake - Query Result Caching
Snowflake - Row Level Security
Snowflake - Schema Evolution
Snowflake - Semi-structured data
Snowflake - Snowpipe
Snowflake - Stages
Snowflake - Streams
Snowflake - Task
Spark
actions transformation
cache vs persist
columns in spark
Deployment mode
Exam Concept
Exam syntax
Join
Join Strategy
MEMORY_AND_DISK vs MEMORY_ONLY StorageLevel
Read Write from SQL DB
Repartition and Coalesce (for optimization)
rows in spark
show take collect limit
Spark Stability
Stage boundary
Storage level (RDD Persistence)
Structured API
Structured API Execution
Structured operations (basic)
Structured operations (Building expression)
Structured operations (diff. data types)
UDF
SQL
special
leetcode SQL50
Clustering vs non-clustering index
Performance Tuning
SQL
SQL leetcode
SQL TYPE
UPSERT Statement
Tableau
Gallery of Visual Elements
Untitled
Untitled (2)
Untitled (2) (3)
[T] vs [sigmoid]
Chart panal
Clickable URL in tooltips
Cohort analysis
create Measure value
Curve Bump Chart
Data model
Data model & approach
Date control
drill down
Exam Prep
Fan chart
Gallery of Overall Layout
Gallery of Special chart
Gallery of Title
Good design & container
Horizontal Bar chart with label
LOD
Make Path without new data set
Order of operation
pill feature
proportional brushing
Regroup (top n else others)
Rounded Bar Charts with Gradient Fill
show axis on the top
Table Calculation
Table with logo
Tableau
Tableau Exchange
Terraform
Terraform
Terraform block - checks
Terraform block - datas
Terraform block - ephemeral
Terraform block - import
Terraform block - locals
Terraform block - module
Terraform block - output
Terraform block - providers
Terraform block - run
Terraform block - terraform
Terraform block - test
Terraform block - variables
Terraform Built-In Function
Terraform import
Terraform meta-argument
Terraform modules
Terraform state
Terraform test
AI
MCP
MCP
Prompting Technique
Temperature
Data Engineering
Aalystics workloads
ACID
CAP theorem
Centipede fact tables
Characteristic of big data - 5 Vs
CI-CD pipeline
Columnar serialization
Concurrency & Parallelism
Conformed Dimensions
Create delta tables
Data Validation and profiling
Hadoop history and evolution
Normalization
outriggers
Page Template
REST API
Schema on writes & on read
SDLC
Shrunken dimensions
Spark
Spark catalog and SQL query
Use delta tables with streaming data
visualization in spark notebook
Work with delta tables in Spark
Markdown Usage
HTML Element
Operation for myself
Page Template
Quartz operation for myself
Other computer concept
exit code
MFA
POSIX
RPC
sort algorithm
Useful window command
Useful window Short cut
Resource
Choice of Chart
My badges
Useful Website
Template
Page Template
Home
❯
⚙️ Stack
❯
Spark
❯
actions transformation
actions transformation
Aug 16, 2025
1 min read
spark
exam
actions: collect, count, fist, take
Graph View
Backlinks
No backlinks found