The operation and maintenance engineer is a hard work in the early stage , In the meantime, I may be working on repairing computers , Pinch the cable , Moving machines , Looks like no status ! Time is also fragmented , All sorts of little things surround you , It's hard to embody personal value , Gradually very confused about the industry , I don't think there is any future .


These dull jobs do make people want , From a technical point of view, these are actually basic skills , It will bring some help to the later operation and maintenance work , Because that's how I've come , Can deeply understand . Therefore, we must maintain a positive attitude in this period , Continuous learning . One day in the future , I believe I will give you back !

okay , Get to the point , According to my years of operation and maintenance work experience , Let's share the learning route of senior operation and maintenance engineers .

primary

1,Linux Basics

You need to be familiar at the beginning Linux/Windows Operating system installation , directory structure , Start up process, etc .

2, system management


Main learning Linux system , In the production environment, the work is basically completed in the character interface , Therefore, we should master dozens of common basic management commands , Including user management , Partition , Software package management , file right , text processing , Process management , Performance analysis tools, etc .

3, Network foundation

OSI and TCP/IP The model must be familiar with . Basic switch , Router concept and implementation principle .

4,Shell Fundamentals of script programming

master Shell Basic grammatical structure , Can write a simple script .

intermediate

1, network service

The most commonly used network services must be deployed , such as vsftp,nfs,samba,bind,dhcp etc. .

Code version management system is indispensable , You can learn from the mainstream SVN and GIT, It's enough to be deployed and easy to use .

Data is often transferred between servers , So you have to be able to use it :rsync and scp.

Data synchronization :inotify/sersync.

Do some work repeatedly , Can be written as a script to run regularly , So you have to be able to configure Linux Timed task service under crond.

2,Web service

Every company basically has a website , Let the website run , It needs to be built Web Service platform .

If it is used PHP Language development , Usually built LAMP,LNMP Website platform , This is the spelling of a technical noun combination , Separately speaking, we must be able to deploy Apache,Nginx,MySQL and PHP.


If it is JAVA Language development , Commonly used Tomcat Running project , In order to improve the access speed , have access to Nginx Reverse proxy Tomcat,Nginx Processing static pages ,Tomcat Processing dynamic pages , Separation of dynamic and static .

It's not that simple to deploy , Also know HTTP How the protocol works , Simple performance tuning .

3, database

Database selection MySQL, It is the most widely used open source database in the world . It must be right to learn it ! And some simple ones SQL sentence , user management , Common storage engines , Database backup and recovery .

Want to go deeper , Master slave replication is required , performance optimization , Mainstream cluster solution :MHA,MGR etc. .NoSQL Of course, it's very popular , Study Redis,MongoDB These two will be fine .

4, security


Safety is important , Don't wait for the system to be compromised , Do security policy again , It was late ! therefore , When a server goes online, it should make security access control strategy immediately , For example, using iptables Only trusted sources are allowed IP visit , Shut down some useless services and ports .

Some common attack types must be known , Otherwise, how can we prescribe the right medicine ! such as CC,DDOS,ARP etc. .

5, monitoring system


Monitoring is essential , It is a life-saving straw for finding and tracing problems in time . You can choose to learn from the mainstream Zabbix Open source monitoring system , Rich functions , It can meet the basic monitoring requirements . Monitoring points include basic server resources , Interface status , Service performance ,PV/UV, Log, etc .

You can also have a dashboard that shows a few key real-time data , such as Grafana, It's going to be cool .

6,Shell Advanced script programming

Shell The script is Linux A powerful tool for automatic work , It has to be written skillfully , So we have to learn more about functions , array , signal , E-mail, etc .

Three swordsmen in text processing (grep,sed,awk) You have to play 6 ah ,Linux Next text processing depends on them .

7,Python Development basis

Shell Scripts do only a few basic tasks , Want to accomplish more complex tasks , For example, call API, Multi process, etc . You need to learn advanced language .


Python It is the most used language in operation and maintenance field , Easy to use , It must be right to learn it ! At this stage, it is enough to master the foundation , For example, the basic grammatical structure , File object operation , function , Iteration object , exception handling , send emails , Database programming, etc .

senior

1,Web Static cache

Users are always calling for slow access to the website , Look at the rich server resources ! Slow site access may not be caused by server resource saturation , There are many factors , For example, the network , Forwarding layers, etc .

For the network , There are North South communication problems , Between visits will be slow , This can be used CDN solve , Cache static pages at the same time , Try to intercept the request at the top level , Reduce back end request and response time .

If not CDN, It can also be used Squid,Varnish,Nginx This kind of caching service implements static page caching , Put it at the flow inlet .

2, colony


The resources of a single server are limited after all , Resistance to high traffic is certainly unsustainable , The key technology to solve this problem is to use load equalizer , Horizontal expansion of multiple sets Web The server , At the same time, it provides services to the outside world , This multiplies performance . The mainstream open source technologies of load balancer include LVS,HAProxy and Nginx. Be familiar with one or two !


Web The server performance bottleneck has been solved , Database is more critical , Or cluster , Take what you learn MySQL Come on , It can be a master multi slave architecture , On this basis, read-write separation , The chief is responsible for writing , Take charge of reading more , Horizontal expansion from library , Another four layer load balancer ahead , Carrying tens of millions of classes PV, Proper !

High availability software also needs to know , A sharp weapon to avoid single point , The mainstream is Keepalived,Heartbeat etc. .


Why are there so many pictures on the website !NFS Shared storage doesn't work , It's a slow process , enjoy tinkering or playing around with ! Distributed file system on , Parallel processing tasks , No single point , High reliability , High performance, etc , The mainstream is FastDFS,MFS,HDFS,Ceph,GFS etc. . At the beginning, I suggest studying FastDFS, It can meet the needs of small and medium-sized enterprises .

3, Virtualization


Hardware server resource utilization is very low , What a waste ! You can virtualize more idle servers , Make many virtual machines , Each virtual machine is a complete operating system . It can greatly improve the utilization rate of resources . It is suggested to learn from open source KVM+OpenStack Cloud platform .

Virtual machine as a basic platform can also , But the elastic scaling of application business is too heavy ! It started for a few minutes , The file is so big , It's too hard to expand quickly !

It's easy to say , upper bowl , The main characteristics of container are rapid deployment and environment isolation . A service is encapsulated in a mirror image , Hundreds of containers can be created in minutes .

Mainstream container technology Docker It must be .


of course , Production environment stand alone Docker In most cases, it is unable to meet the business requirements , Can be deployed Kubernetes,Swarm Cluster management container , Form a large resource pool , centralized management , Provide strong support for infrastructure .

4, automation

Repeated work , Not only can't we improve efficiency , Value is not reflected .

Standardization of all operation and maintenance work , For example, the environment version , directory structure , Unified operating system, etc . On the basis of standardization, more automation can be achieved , Click the mouse or click a few commands to complete a complex task , It's so cool !

therefore , All operations are as automated as possible , Reduce human error , improve work efficiency .

Mainstream server centralized management tools :Ansible,Saltstack

Choose either of these two .

Continuous integration tools :Jenkins

5,Python Advanced development

We can learn further Python development , Master object oriented programming .


It's better to learn one too Web Framework development website , for example Django,Flask, It mainly develops the operation and maintenance management system , Write some complex processes to the platform , Re integration of centralized management tools , It can create a management platform belonging to operation and maintenance .

6, Log analysis system

The journal is also important , Regular analysis , Potential hazards can be found , Extract something of value .

An open source log system :ELK

Learn to deploy and use , Provide log view requirements for development .

7, performance optimization

Deployment is not enough , Performance optimization can maximize service load .

This one is also more difficult , It's also one of the keys to high pay , I have to study hard for money !

From the hardware layer , Operating system layer , Thinking about software layer and architecture layer .

Weapon type Weapon name
text processing grep,sed,awk
data transmission Rsync,Scp,Inodify/Sersync
Process management Supervisor
performance analysis top,free,df,iftop,iostat,vmstat,dstat,sar,sysdig
network service vsftp,nfs,samba,bind,dhcp,postfix
Web service Apache,Nginx,Tomcat,JBoss,Resin
database MySQL,MariaDB,Percona Server for MySQL
NoSQL Redis,MongoDB
Message middleware RabbitMQ,ActiveMQ
version management SVN,Git
Static cache Squid,Varnish,Nginx
load balancing LVS,HAProxy,Nginx
High availability software Keepalived,Heartbeat,DRBD,corosync+pacemaker
Centralized management tools Ansible,Saltstack,Chef,Puppet
Virtualization and cloud platform KVM,Xen,Docker,Kubernetes,Openstack,Cloudstack
Automatic installation Kickstart,Cobbler
Bag capture analysis Tcpdump,Wireshark
Continuous integration Jenkins,Gitlab
MySQL agent Altas,Cobar,Mycat
Pressure measurement ab,fio,sysbench,mysqlslap
security check chrootkit,rkhunter
Log system ELK
monitor Zabbix,Cacti,Nagios,Grafana
consciousness

1, insist

Learning is a long process , It's a career that each of us needs to stick to for a lifetime .

Persistence is the key , It's hard to insist , Success in persistence !

2, target

No work without goals , What is not quantified is not a goal .

Every stage , Set a goal .

such as : Set a small goal that you can achieve , Make it 100 million !

3, share

Learn to share , The value of technology lies in the effective transfer of knowledge to the outside world , Let more people know about it .

As long as everyone brings out something , Think about what it's going to be like ?

Right direction , I'm not afraid of a long way to go !

Ten items Linux common sense

1,GNU and GPL

GNU plan ( Also known as the Gino project ), By Richard
Stallman( Richard · Stallman ) stay 1983 year 9 month 27 Free software collective cooperation program launched in Japan . Its goal is to create a completely free operating system .GNU Also known as free software engineering project .

GPL yes GNU General public license for (GNU General Public
License,GPL), Namely “ Anti copyright ” concept , yes GNU Agreement one , The purpose is to protect GNU Software can be used freely , copy , Research , Modification and release . At the same time, the software must be released in the form of source code .

GNU System and Linux The kernel combines to form a complete operating system : One is based on Linux Of GNU system , This operating system is commonly referred to as “GNU/Linux”, Or abbreviation Linux.

2,Linux Release

A typical Linux The distribution includes :Linux kernel , some GNU Libraries and tools , command line shell, Graphical interface X
Window System and corresponding desktop environment , as KDE or GNOME, And contains thousands of different office kits , compiler , Application software from text editor to scientific tool .

Mainstream distribution :

Red Hat Enterprise Linux,CentOS,SUSE,Ubuntu,Debian,Fedora,Gentoo

3,Unix and Linux


Linux Is based on Unix Of , belong to Unix class ,Uinx The operating system supports multiple users , Multitasking , Multithreading and multiple support CPU Architecture operating system .Linux Inherited Unix Design idea with network as the core , It is a stable multi-user network operating system .

4,Swap partition


Swap partition , The exchange area , When the system runs out of physical memory , And Swap Exchange . When the physical memory of the system is insufficient , Free up some space in the hard disk , To be used by the currently running program . When those programs are going to run , From Swap Restore the saved data in the partition to memory . Those programs that have been released memory space are usually programs that have not operated for a long time .

Swap Generally, the space should be greater than or equal to the size of the physical memory , At the same time, the minimum should not be less than 64M, The maximum should be twice the physical memory .

5,GRUB The concept of

GNU GRUB(GRand Unified Bootloader abbreviation “GRUB”) It's one from GNU Project's multi operating system boot manager .


GRUB Is a boot manager that supports multiple operating systems , In a computer with multiple operating systems , It can be done through GRUB Select the operating system that the user wants to run when the computer starts . meanwhile GRUB Can guide Linux Different cores on system partition , It can also be used to pass boot parameters to the kernel , If entering single user mode .

6,Buffer and Cache


Cache( cache ) be located CPU Temporary storage between and memory , The cache size is much smaller than memory, but the switching speed is much faster than memory .Cache By caching file data blocks , solve CPU The contradiction between operation speed and memory read / write speed , increase CPU Data exchange between and memory 速度.Cache缓存越大,CPU处理速度越快.


Buffer(缓冲)高速缓冲存储器,通过缓存磁盘(I/O设备)数据块,加快对磁盘上数据的访问,减少I/O,提高内存和硬盘(或其他I/O设备)之间的数据交换速度.Buffer是即将要被写入磁盘的,而Cache是被从磁盘中读出来的.

7,TCP三次握手

(1)请求端发送SYN(SYN=A)数据包,等待响应端确认

(2)响应端接收SYN,并返回SYN(A+1)和自己的ACK(K)包给请求端

(3)请求端接收到响应端的SYN+ACK包,再次向响应端发送确认包ACK(K+1)

请求端和响应端建立TCP连接,完成三次握手,开始进行数据传输.

8,linux系统目录结构


Linux文件系统采用带链接的树形目录结构,即只有一个根目录(通常用“/”表示),其中含有下级子目录或文件的信息;子目录中又可含有更下级的子目录或者文件的信息.

*
/:第一层次结构的根,整个文件系统层次结构的根目录.即文件系统的入口,最高一级目录.

*
/boot:包含Linux内核及系统引导程序所需的文件,例如kernel,initrd;grub系统引导管理器也在这个目录下.

*
/bin:基本系统所需要的命令,功能和"/usr/bin"类似,这个目录下的文件都是可执行的.普通用户也是可以执行的.

*
/sbin:基本的系统维护命令,只能由超级用户使用.

*
/etc:所有的系统配置文件.

*
/dev:设备文件存储目录.像终端,磁盘,光驱等.

*
/var:存放经常变动的数据,像日志,邮件等.

*
/home:普通用户的目录默认存储目录.

*
/opt:第三方软件的存放目录,比如用户自定义软件包和编译的软件包就安装到这个目录中.

*
/lib:库文件和内核模块存放目录,包含系统程序所需要的所有共享库文件.

9,硬链接和软链接

硬链接(Hard Link):硬链接是使用同一个索引节点(inode号)的链接,
即可以允许多个文件名指向同一个文件索引节点(硬链接不支持目录链接,不能跨分区链接),删除一个硬链接,不会影响该索引节点的源文件以及其下的多个硬链接.

ln source new-link

软连接(符号链接,Symbolic
Link):符号链接是以路径的形式创建的链接,类似于windows的快捷方式链接,符号链接允许创建多个文件名链接到同一个源文件,删除源文件,其下的所有软连接将不可用.(软连接支持目录,支持跨分区,跨文件系统)

ln -s source new-link

10,RAID技术

磁盘阵列(Redundant Arrays of independent Disks,RAID),廉价冗余(独立)磁盘阵列.


RAID是一种把多块独立的物理硬盘按不同的方式组合起来形成一个硬盘组(逻辑硬盘),提供比单个硬盘更高的存储性能和数据备份技术.RAID技术,可以实现把多个磁盘组合在一起作为一个逻辑卷提供磁盘跨越功能;可以把数据分成多个数据块(Block)并行写入/读出多个磁盘以提高访问磁盘的速度;可以通过镜像或校验操作提供容错能力.具体的功能以不同的RAID组合实现.


在用户看来,RAID组成的磁盘组就像是一个硬盘,可以对它进行分区,格式化等操作.RAID的存储速度比单个硬盘高很多,并且可以提供自动数据备份,提供良好的容错能力.

RAID级别,不同的RAID组合方式分为不同的RAID级别:

*
RAID
0:称为Stripping条带存储技术,所有磁盘完全地并行读,并行写,是组建磁盘阵列最简单的一种形式,只需要2块以上的硬盘即可,成本低,可以提供整个磁盘的性能和吞吐量,但RAID
0没有提供数据冗余和错误修复功能,因此单块硬盘的损坏会导致所有的数据丢失.(RAID
0只是单纯地提高磁盘容量和性能,没有为数据提供可靠性保证,适用于对数据安全性要求不高的环境)

*
RAID 1:镜像存储,通过把两块磁盘中的一块磁盘的数据镜像到另一块磁盘上,
实现数据冗余,在两块磁盘上产生互为备份的数据,其容量仅等于一块磁盘的容量.当数据在写入一块磁盘时,会在另一块闲置的磁盘上生产镜像,在不影响性能情况下最大限度的保证系统的可靠性和可修复性;当原始数据繁忙时,可直接从镜像拷贝中读取数据(从两块硬盘中较快的一块中读出),提高读取性能.相反的,RAID
1的写入速度较缓慢.RAID 1一般支持“热交换”,即阵列中硬盘的移除或替换可以在系统运行状态下进行,无须中断退出系统.RAID
1是磁盘阵列中硬盘单位成本最高的,但它提供了很高的数据安全性,可靠性和可用性,当一块硬盘失效时,系统可以自动切换到镜像磁盘上读写,而不需要重组失效的数据.

*
RAID 0+1:也被称为RAID 10,实际是将RAID 0和RAID
1结合的形式,在连续地以位或字节为单位分割数据并且并行读/写多个磁盘的同时,为每一块磁盘做镜像进行冗余.通过RAID
0+1的组合形式,数据除分布在多个盘上外,每个盘都有其物理镜像盘,提供冗余能力,允许一个以下磁盘故障,而不影响数据可用性,并且有快速读/写能力.RAID
0+1至少需要4个硬盘在磁盘镜像中建立带区集.RAID 0+1技术在保证数据高可靠性的同时,也保证了数据读/写的高效性.

*
RAID 5:是一种存储性能,数据安全和存储成本兼顾的存储解决方案.RAID 5可以理解为是RAID 0和RAID 1的折衷方案,RAID
5至少需要三块硬盘.RAID 5可以为系统提供数据安全保障,但保障程度要比镜像低而磁盘空间利用率要比镜像高.RAID 5具有和RAID
0相近似的数据读取速度,只是多了一个奇偶校验信息,写入数据的速度比对单个磁盘进行写入操作稍慢.同时由于多个数据对应一个奇偶校验信息,RAID
5的磁盘空间利用率要比RAID 1高,存储成本相对较低,是目前运用较多的一种解决方案.

                                                                             
                    转自          计算机与网络安全
<http://mp.sohu.com/profile?xpt=Y29tcHV0ZXItbmV0d29ya0Bzb2h1LmNvbQ==&_f=index_pagemp_1>