The operation and maintenance engineer is a hard work in the early stage , In the meantime, I may be working on repairing computers , Pinch the cable , Moving machines , Looks like no status ! Time is also fragmented , All sorts of little things surround you , It's hard to embody personal value , Gradually very confused about the industry , I don't think there is any future .
These dull jobs do make people want , From a technical point of view, these are actually basic skills , It will bring some help to the later operation and maintenance work , Because that's how I've come , Can deeply understand . Therefore, we must maintain a positive attitude in this period , Continuous learning . One day in the future , I believe I will give you back !
okay , Get to the point , According to my years of operation and maintenance work experience , Let's share the learning route of senior operation and maintenance engineers .
You need to be familiar at the beginning Linux/Windows Operating system installation , directory structure , Start up process, etc .
2, system management
Main learning Linux system , In the production environment, the work is basically completed in the character interface , Therefore, we should master dozens of common basic management commands , Including user management , Partition , Software package management , file right , text processing , Process management , Performance analysis tools, etc .
3, Network foundation
OSI and TCP/IP The model must be familiar with . Basic switch , Router concept and implementation principle .
4,Shell Fundamentals of script programming
master Shell Basic grammatical structure , Can write a simple script .
1, network service
The most commonly used network services must be deployed , such as vsftp,nfs,samba,bind,dhcp etc. .
Code version management system is indispensable , You can learn from the mainstream SVN and GIT, It's enough to be deployed and easy to use .
Data is often transferred between servers , So you have to be able to use it ：rsync and scp.
Data synchronization ：inotify/sersync.
Do some work repeatedly , Can be written as a script to run regularly , So you have to be able to configure Linux Timed task service under crond.
Every company basically has a website , Let the website run , It needs to be built Web Service platform .
If it is used PHP Language development , Usually built LAMP,LNMP Website platform , This is the spelling of a technical noun combination , Separately speaking, we must be able to deploy Apache,Nginx,MySQL and PHP.
If it is JAVA Language development , Commonly used Tomcat Running project , In order to improve the access speed , have access to Nginx Reverse proxy Tomcat,Nginx Processing static pages ,Tomcat Processing dynamic pages , Separation of dynamic and static .
It's not that simple to deploy , Also know HTTP How the protocol works , Simple performance tuning .
Database selection MySQL, It is the most widely used open source database in the world . It must be right to learn it ! And some simple ones SQL sentence , user management , Common storage engines , Database backup and recovery .
Want to go deeper , Master slave replication is required , performance optimization , Mainstream cluster solution ：MHA,MGR etc. .NoSQL Of course, it's very popular , Study Redis,MongoDB These two will be fine .
Safety is important , Don't wait for the system to be compromised , Do security policy again , It was late ! therefore , When a server goes online, it should make security access control strategy immediately , For example, using iptables Only trusted sources are allowed IP visit , Shut down some useless services and ports .
Some common attack types must be known , Otherwise, how can we prescribe the right medicine ! such as CC,DDOS,ARP etc. .
5, monitoring system
Monitoring is essential , It is a life-saving straw for finding and tracing problems in time . You can choose to learn from the mainstream Zabbix Open source monitoring system , Rich functions , It can meet the basic monitoring requirements . Monitoring points include basic server resources , Interface status , Service performance ,PV/UV, Log, etc .
You can also have a dashboard that shows a few key real-time data , such as Grafana, It's going to be cool .
6,Shell Advanced script programming
Shell The script is Linux A powerful tool for automatic work , It has to be written skillfully , So we have to learn more about functions , array , signal , E-mail, etc .
Three swordsmen in text processing （grep,sed,awk） You have to play 6 ah ,Linux Next text processing depends on them .
7,Python Development basis
Shell Scripts do only a few basic tasks , Want to accomplish more complex tasks , For example, call API, Multi process, etc . You need to learn advanced language .
Python It is the most used language in operation and maintenance field , Easy to use , It must be right to learn it ! At this stage, it is enough to master the foundation , For example, the basic grammatical structure , File object operation , function , Iteration object , exception handling , send emails , Database programming, etc .
1,Web Static cache
Users are always calling for slow access to the website , Look at the rich server resources ! Slow site access may not be caused by server resource saturation , There are many factors , For example, the network , Forwarding layers, etc .
For the network , There are North South communication problems , Between visits will be slow , This can be used CDN solve , Cache static pages at the same time , Try to intercept the request at the top level , Reduce back end request and response time .
If not CDN, It can also be used Squid,Varnish,Nginx This kind of caching service implements static page caching , Put it at the flow inlet .
The resources of a single server are limited after all , Resistance to high traffic is certainly unsustainable , The key technology to solve this problem is to use load equalizer , Horizontal expansion of multiple sets Web The server , At the same time, it provides services to the outside world , This multiplies performance . The mainstream open source technologies of load balancer include LVS,HAProxy and Nginx. Be familiar with one or two !
Web The server performance bottleneck has been solved , Database is more critical , Or cluster , Take what you learn MySQL Come on , It can be a master multi slave architecture , On this basis, read-write separation , The chief is responsible for writing , Take charge of reading more , Horizontal expansion from library , Another four layer load balancer ahead , Carrying tens of millions of classes PV, Proper !
High availability software also needs to know , A sharp weapon to avoid single point , The mainstream is Keepalived,Heartbeat etc. .
Why are there so many pictures on the website !NFS Shared storage doesn't work , It's a slow process , enjoy tinkering or playing around with ! Distributed file system on , Parallel processing tasks , No single point , High reliability , High performance, etc , The mainstream is FastDFS,MFS,HDFS,Ceph,GFS etc. . At the beginning, I suggest studying FastDFS, It can meet the needs of small and medium-sized enterprises .
Hardware server resource utilization is very low , What a waste ! You can virtualize more idle servers , Make many virtual machines , Each virtual machine is a complete operating system . It can greatly improve the utilization rate of resources . It is suggested to learn from open source KVM+OpenStack Cloud platform .
Virtual machine as a basic platform can also , But the elastic scaling of application business is too heavy ! It started for a few minutes , The file is so big , It's too hard to expand quickly !
It's easy to say , upper bowl , The main characteristics of container are rapid deployment and environment isolation . A service is encapsulated in a mirror image , Hundreds of containers can be created in minutes .
Mainstream container technology Docker It must be .
of course , Production environment stand alone Docker In most cases, it is unable to meet the business requirements , Can be deployed Kubernetes,Swarm Cluster management container , Form a large resource pool , centralized management , Provide strong support for infrastructure .
Repeated work , Not only can't we improve efficiency , Value is not reflected .
Standardization of all operation and maintenance work , For example, the environment version , directory structure , Unified operating system, etc . On the basis of standardization, more automation can be achieved , Click the mouse or click a few commands to complete a complex task , It's so cool !
therefore , All operations are as automated as possible , Reduce human error , improve work efficiency .
Mainstream server centralized management tools ：Ansible,Saltstack
Choose either of these two .
Continuous integration tools ：Jenkins
5,Python Advanced development
We can learn further Python development , Master object oriented programming .
It's better to learn one too Web Framework development website , for example Django,Flask, It mainly develops the operation and maintenance management system , Write some complex processes to the platform , Re integration of centralized management tools , It can create a management platform belonging to operation and maintenance .
6, Log analysis system
The journal is also important , Regular analysis , Potential hazards can be found , Extract something of value .
An open source log system ：ELK
Learn to deploy and use , Provide log view requirements for development .
7, performance optimization
Deployment is not enough , Performance optimization can maximize service load .
This one is also more difficult , It's also one of the keys to high pay , I have to study hard for money !
From the hardware layer , Operating system layer , Thinking about software layer and architecture layer .
Weapon type Weapon name
text processing grep,sed,awk
data transmission Rsync,Scp,Inodify/Sersync
Process management Supervisor
performance analysis top,free,df,iftop,iostat,vmstat,dstat,sar,sysdig
network service vsftp,nfs,samba,bind,dhcp,postfix
Web service Apache,Nginx,Tomcat,JBoss,Resin
database MySQL,MariaDB,Percona Server for MySQL
Message middleware RabbitMQ,ActiveMQ
version management SVN,Git
Static cache Squid,Varnish,Nginx
load balancing LVS,HAProxy,Nginx
High availability software Keepalived,Heartbeat,DRBD,corosync+pacemaker
Centralized management tools Ansible,Saltstack,Chef,Puppet
Virtualization and cloud platform KVM,Xen,Docker,Kubernetes,Openstack,Cloudstack
Automatic installation Kickstart,Cobbler
Bag capture analysis Tcpdump,Wireshark
Continuous integration Jenkins,Gitlab
MySQL agent Altas,Cobar,Mycat
Pressure measurement ab,fio,sysbench,mysqlslap
security check chrootkit,rkhunter
Log system ELK
Learning is a long process , It's a career that each of us needs to stick to for a lifetime .
Persistence is the key , It's hard to insist , Success in persistence !
No work without goals , What is not quantified is not a goal .
Every stage , Set a goal .
such as ： Set a small goal that you can achieve , Make it 100 million !
Learn to share , The value of technology lies in the effective transfer of knowledge to the outside world , Let more people know about it .
As long as everyone brings out something , Think about what it's going to be like ?
Right direction , I'm not afraid of a long way to go !
Ten items Linux common sense
1,GNU and GPL
GNU plan （ Also known as the Gino project ）, By Richard
Stallman（ Richard · Stallman ） stay 1983 year 9 month 27 Free software collective cooperation program launched in Japan . Its goal is to create a completely free operating system .GNU Also known as free software engineering project .
GPL yes GNU General public license for （GNU General Public
License,GPL）, Namely “ Anti copyright ” concept , yes GNU Agreement one , The purpose is to protect GNU Software can be used freely , copy , Research , Modification and release . At the same time, the software must be released in the form of source code .
GNU System and Linux The kernel combines to form a complete operating system ： One is based on Linux Of GNU system , This operating system is commonly referred to as “GNU/Linux”, Or abbreviation Linux.
A typical Linux The distribution includes ：Linux kernel , some GNU Libraries and tools , command line shell, Graphical interface X
Window System and corresponding desktop environment , as KDE or GNOME, And contains thousands of different office kits , compiler , Application software from text editor to scientific tool .
Mainstream distribution ：
Red Hat Enterprise Linux,CentOS,SUSE,Ubuntu,Debian,Fedora,Gentoo
3,Unix and Linux
Linux Is based on Unix Of , belong to Unix class ,Uinx The operating system supports multiple users , Multitasking , Multithreading and multiple support CPU Architecture operating system .Linux Inherited Unix Design idea with network as the core , It is a stable multi-user network operating system .
Swap partition , The exchange area , When the system runs out of physical memory , And Swap Exchange . When the physical memory of the system is insufficient , Free up some space in the hard disk , To be used by the currently running program . When those programs are going to run , From Swap Restore the saved data in the partition to memory . Those programs that have been released memory space are usually programs that have not operated for a long time .
Swap Generally, the space should be greater than or equal to the size of the physical memory , At the same time, the minimum should not be less than 64M, The maximum should be twice the physical memory .
5,GRUB The concept of
GNU GRUB（GRand Unified Bootloader abbreviation “GRUB”） It's one from GNU Project's multi operating system boot manager .
GRUB Is a boot manager that supports multiple operating systems , In a computer with multiple operating systems , It can be done through GRUB Select the operating system that the user wants to run when the computer starts . meanwhile GRUB Can guide Linux Different cores on system partition , It can also be used to pass boot parameters to the kernel , If entering single user mode .
6,Buffer and Cache
Cache（ cache ） be located CPU Temporary storage between and memory , The cache size is much smaller than memory, but the switching speed is much faster than memory .Cache By caching file data blocks , solve CPU The contradiction between operation speed and memory read / write speed , increase CPU Data exchange between and memory 速度.Cache缓存越大,CPU处理速度越快.
ln source new-link
ln -s source new-link
磁盘阵列（Redundant Arrays of independent Disks,RAID）,廉价冗余（独立）磁盘阵列.
RAID 0+1：也被称为RAID 10,实际是将RAID 0和RAID
RAID 5：是一种存储性能,数据安全和存储成本兼顾的存储解决方案.RAID 5可以理解为是RAID 0和RAID 1的折衷方案,RAID
5至少需要三块硬盘.RAID 5可以为系统提供数据安全保障,但保障程度要比镜像低而磁盘空间利用率要比镜像高.RAID 5具有和RAID