Decompression

1
2
3
4
5
6
7
8
9
10
11

tar -xvf zookeeper-3.5.2-alpha.tar.gz
cp zoo_sample.cfg zoo.cfg

vim zoo.cfg
# modify config
admin.serverPort=8081
dataDir=/home/xxx/zookeeper-3.5.2-alpha/data  

# check java version
echo $JAVA_HOME
java -version

Test

1
2
3
4
5
6
7
8
9
10
11
12
13

./zkServer.sh start
# check
ps -ef|grep zookeeper

# if no progress
./zkServer.sh start-foreground

# shell client
./zkCli.sh
...
[zk: localhost:2181(CONNECTED) 0] create /key "hello zookeeper"
Created /key
[zk: localhost:2181(CONNECTED) 1] quit

Create structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

.
├── pom.xml
├── src/
│   ├── main/
│   │   ├── com/
│   │   │   └── leon/
│   │   │       └── Hello.java
│   │   └── resources/
│   └── test/
│       ├── java/
│       └── resources/
└── web/
    ├── WEB-INF/
    │   └── web.xml
    └── hello.jsp

pom.xml web.xml

add dependency

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.leon</groupId>
  <artifactId>hello</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>hello Maven Webapp</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>javax.servlet-api</artifactId>
      <version>3.1.0</version>
      <scope>provided</scope>
    </dependency>
  </dependencies>
  <build>
    <finalName>hello</finalName>
  </build>
</project>

add app config

1
2
3
4
5
6

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         version="3.1">
</web-app>

Servlet

container action

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

package com.leon;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

@WebServlet(name = "hello", urlPatterns = {"/"})
public class Hello extends javax.servlet.http.HttpServlet {

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        resp.getWriter().print("Hello World");
    }

    @Override
    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        doGet(req, resp);
    }

}

IntelliJ IDEA

1. Open Module Setting
2. Set Project SDK/ Product/ Artifact direcotry
3. Set Module resource folder, compile dependencies.
4. Set Facets/ Artifact

test

http://localhost:8080/

1
2
3
4
5
6
7
8
9
10

CREATE TABLE `user_org` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(30)  DEFAULT NULL,
  `organizations` varchar(60)  DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB

insert into user_org (name, organizations) values('Betty', '1001,1002,1003');
insert into user_org (name, organizations) values('William', '1001,1002');
insert into user_org (name, organizations) values('Thomas', '1002,1003');

Mysql

User find_in_set function directly.

1

select * from user_org where find_in_set('1003', organizations);

Oracle

1. Use LIKE key word:

   1	select * from user_org where organizations like '%1003%';

this may mis match, if element length are not fixed.

1. Create a function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

CREATE OR REPLACE
FUNCTION FIND_IN_SET(piv_str1 varchar2, piv_str2 varchar2, p_sep varchar2 := ',')
RETURN NUMBER IS
  l_idx       number:=0;      -- 用于计算piv_str2中分隔符的位置
  str         varchar2(500);  -- 根据分隔符截取的子字符串
  piv_str     varchar2(500) := piv_str2; -- 将piv_str2赋值给piv_str
  res         number:=0;      -- 返回结果
  res_place   number:=0;      -- 原字符串在目标字符串中的位置
BEGIN
-- 如果piv_str中没有分割符，直接判断piv_str1和piv_str是否相等，相等 res_place=1
IF instr(piv_str, p_sep, 1) = 0THEN
   IF piv_str = piv_str1 THEN
      res_place:=1;
   END IF;
ELSE
 -- 循环按分隔符截取piv_str
LOOP
    l_idx := instr(piv_str,p_sep);
    --
    res_place := res_place + 1;
    -- 当piv_str中还有分隔符时
      IF l_idx > 0THEN
      -- 截取第一个分隔符前的字段str
         str:= substr(piv_str,1,l_idx-1);
         -- 判断 str 和piv_str1 是否相等，相等则结束循环判断
         IF str = piv_str1 THEN
           res:= res_place;
           EXIT;
         END IF;
        piv_str := substr(piv_str,l_idx+length(p_sep));
      ELSE
      -- 当截取后的piv_str 中不存在分割符时，判断piv_str和piv_str1是否相等，相等 res=res_path
        IF piv_str = piv_str1 THEN
           res:= res_place;
        END IF;
        -- no matter what jump out
        EXIT;
      END IF;
 END LOOP;
 -- stoped
 END IF;
 res := res_place;
 -- return
 RETURN res;
END FIND_IN_SET;

ibatis3.x change name to mybatis.

Object relation mapping optimize

mybatis could package N+1 query problem, in <association> and <collection>
ibatis2 use nest query is a brute method, we can package in DAO or service as will.

But mybatis N+1 query not support paging query.
May be 1 to many is difficult to implements in framework,
and the number you want can’t be predict.

Mybatis implements interface binding

In ibatis2.x, we need to write
which xml mapping file is the DAO implements class binding.

In mybatis DAO interface’s name automatically binding
implements with xml file.

ps. although mybatis support annotation way config DAO,
but it loose flexibility, and invade code too much.

Configuration

In config

1. dtd restraint file is different.
2. ibatis root element is sqlMapConfig, mybatis is configuration
3. global setting properties is different.
4. ibatis use sqlMap，mybatis use mappers.

In mapping table

1. in ibatis namespace is not necessary, and has no meaning.
   in mybatis it mean file’s corresponding DAO interface.

2. ibatis has resultMap resultClass two return types
   resultMap mean self define class, resultClass nean java class.
   mybatis union these too resultType.

3. ibatis use parameterClass, mybatis use parameterType

4. parameter difference
   mybatis use OGNL expression.

   1 2 3 4 5

   # ibatis WHERE ID = #id:VARCHAR# # mybatis where ID = #{id,jdbcType=VARCHAR}

CLR

ibatis use <procedure> tag call precedure process.

1
2
3
4
5
6
7
8
9

<parameterMap id="reduceParameters" class="map">
<parameter property="id" jdbcType="INTEGER" javaType="java.lang.Integer" mode="IN"/>
<parameter property="firstName" jdbcType="VARCHAR" javaType="java.lang.String" mode="IN"/>
<parameter property="lastName" jdbcType="VARCHAR" javaType="java.lang.String" mode="IN"/>
</parameterMap>

<procedure id="reduceGroupNumber" parameterMap="reduceParameters">
{call reduce_group_number (?, ?, ?)}
</procedure>

In mybatis <proccedure> has been removed,
<select> <insert> <update> replaced.

1
2
3

<select id="reduceGroupNumber" parameterMap="reduceParameters" statementType="CALLABLE">
{ ? = call reduce_group_number (?,?,?)}
</select>

statementType="CALLABLE" show it’s not a usual sql request.

Integration difference

ibatis:

1
2
3
4

<bean id="sqlMapClient" class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
    <property name="configLocation" value="classpath:sqlMap-config.xml" />
    <property name="dataSource" ref="dataSource" />
</bean>

mybatis:

1
2
3
4
5

<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
    <property name="dataSource" ref="dataSource" />
    <property name="configLocation" value="classpath:com/leon/core/config/mybatis-config.xml" />  
    <property name="mapperLocations" value="classpath*:com/leon/**/*Mapper.xml" />
</bean>

1. implements machine from SqlMapClient to SqlSessionFactory
2. class processor interface from TypeHandlerCallback to TypeHandler
3. DataSourceFactory move to org.apache.ibatis.datasource package

ORA-00257: archiver error. Connect internal only, until freed

1. login in sys
sqlplus sys/pass@tt as sysdba

2. find archiv log position
   SQL> show parameter log_archive_dest;

3. if value is null, check directory
   SQL> archive log list;

4. check flash recovery area use condition
   SQL> select * from V$FLASH_RECOVERY_AREA_USAGE;

5. find recovery directory
   SQL> show parameter recover;

6. remove log file in no use.
   caution: reserver file remain in time.

7. rman maintain
   rman target sys/pass

8. check archivelog in no use
   RMAN> crosscheck archivelog all;

9. delete expired file in db
   RMAN> delete expired archivelog all;
RMAN> delete archivelog until time 'sysdate-1';

10. check again
    SQL> select * from V$FLASH_RECOVERY_AREA_USAGE;

Others

if archive log model can’t startup,
recover to noarchive log startup then shutdown.

1
2
3
4
5

shutdown immediate;
startup mount;
alter database noarchivelog;
alter database open;
shutdown immediate;

turn to archive log again

1
2
3
4
5
6

shutdown immediate;
startup mount;
show parameter log_archive_dest;
alter database archivelog;
archive log list;
alter database open;

If still can’t modify startup.
Try to delete more log or expand space.

1
2
3
4
5
6
7
8
9
10
11
12
13

## check log group
SQL> select group#,sequence# from v$log;

## clean more space
SQL> alter database clear unarchived logfile group 1;
alter database open;

## change storage position
select name from v$datafile;
alter system set log_archive_dest='/opt/app/oracle/oradata/usagedb/arch' scope=spfile

## expand space
SQL> alter system set db_recovery_file_dest_size=3G scope=both;

For example, when a sheet have about 1 million record need import in to database.
How to do this.

1
2
3
4
5
6
7

insert into zoo_sea (id, name, age, birthday, status)
  select 1, 'dog', '2017-05-01', 1 from dual
union all select 2, 'dog', '2017-05-02', 2 from dual
union all select 3, 'dog', '2017-05-03', 1 from dual
union all select 4, 'dog', '2017-05-04', 3 from dual
union all select 5, 'dog', '2017-05-05', 2 from dual
...

make 1 million data into 10000 sql command, insert can finish with in 20 minutes.

Oracle is different with Mysql or sqlServer, it need client connect to DB server. But mysql or sqlServer can connect DB server with specific ip connect.

Install oracle instance client

open Net Manager Oracle Net Config Local Service Naming
click add config new net service.

It actually add a config in this file:
E:\Oracle\product\10.2.0\client_1\NETWORK\ADMIN\tnsnames.ora

1
2
3
4
5
6
7
8
9
10

ORCL =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 127.0.0.1)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = ORCL)
    )
  )

Problem 1

If install server and client in one machine, server directory will have same config file.

D:\Oracle\product\10.2.0\db_1\NETWORK\ADMIN\tnsnames.ora

So when config done Net Manager connnet to oracle, ORA-12154 may happen.
Then you should doubt, which config PL/SQL use.
In this time use tnsping command.

1. copy config file content, make two file synchronized.
2. move path position, put client before db service.

This problem coursed by install sequence
If install DB service first, Client next, then this problem will not happen**

Problem 2

If PL/SQL’s insatll directory have () like 64bit system Program Files (x86),
then no matter how you adjust config, it always alert ORA-12154.
So check software’s insatll path can avoid this problem.

IO Model

Memcached is multi-thread, not block reuse IO net model.
it has main listen thread main worker sub thread.
‘leader’ listened net connnect, dispatch to worker thread.
In network layer, memcached use libevent package event.
But this bring concurrency and lock problem, like stats command.

Redis reuse single thread IO model, it’s self package a AeEvent event framework.
It implements epoll kqueue select event,
for simple IO operation, single thread can reach the top speed.
But for some calc operation, this affect global throughput alot.

Memory manage

Memcached use prepare way to manage memory.
In this way memory pool can avoid the cost in apply and release memory.

Redis use real-time way to manage memory.

Data consistency

Memcached provide cas command, it can make sure data consistency.

Redis has no cas command, but it provide transaction function.
It make sure series of commands’ atomicity, it wouldn’t be interrupted.

Storage way

Memcached only support key-value storage,
not support enumerate / persistence/ copy and so on function.

Redis support except key-value way, but list set sorted set hash.
In other hand, redis support data persistence, copy and so on function.

Summary

1. When data in-memory use Redis is better.

2. Redis in alot of place play a role in replace Memcached.

3. When need other data structure, Redis is better.

4. When storage data can’t be remove, Redis is better.

Create some ghost memory

1. Create a long exist thread or thread pool.
2. Load a class through ClassLoader.
3. Distribute a big block memory to a static strong reference.
   Then use ThreadLocal storage [thread reference?] or [object reference?]
4. Thread clean selfdefine class or load this class’s loader.
5. repeat these steps.

Some mistake

1. static reference hold container.

   1 2 3

   class Holder { static final List list = new ArrayList(100); }

2. call long String’s intern() method

   1 2	String src = "......"; str.intern();

3. forgot release resource

   1 2 3 4 5 6 7

   try { BufferedReader br = new BufferedReader(new FileReader("...")); ... ... } catch (Exception e) { e.printStacktrace(); }

4. forgot close door

   1 2 3 4 5 6 7

   try { Connection conn = ConnectionFactory.getConnection(); ... ... } catch (Exception e) { e.printStacktrace(); }

5. GC can’t reach like native true memory

6. Web container in application context
   getServletContext().setAttribute("rubbish", map);
   no reboot and not remove rubbish.

7. Web container in session context
   session.setAttribute("rubbish", map);
   no reboot / not invalid and not remove rubbish.

8. Not right JVM option
   IBM JDK noclassgc option, it stop no use class’s garbage collect.

Bean not define properly

If HashSet’s element object have no standard implement
or no implement of equals() and hashCode().
Collection can’t ignore repeated element.

Thread relevant

1. Create but not start thread.

2. Thread inherited ContextClassLoader or ccessControlContext,
   and use ThreadGroup or InheritedThreadLocal.
   They affect java.util.concurrent.Executor

3. Not use ThreadLocal as cache unless truly necessary.
   If thread not in it’s life cycle, load ContextClassLoader, gc can’t reach it.

4. When ThreadGroup have no subThread
   but have sub ThreadGroup call ThreadGroup.destroy().
   It lead to this sub ThreadGroup can’t remove from it’s parent ThreadGroup.

5. use WeakHashMap, value hold key reference..

Introduce

NIO is make up for IO‘s lack

NIO have some new characters:
no block i/o, selector, buffer and channel
Channel Buffer Selector is the main force.

• Channel like Stream in traditional style java.

• Buffer

  1 2 3 4 5 6 7

  ByteBuffer byte CharBuffer char ShortBuffer short IntBuffer int LongBuffer long FloatBuffer float DoubleBuffer double

• Selector
  Used to listen mulity channel’s event.
  In traditional tyle when i/o unblock, we know we can i/o.
  But use NIO, we need this equipment to manage i/o.

The difference

1. IO is design for stream, NIO is design for block.
   IO stream operate a byte every time, it lead to low efficiency.

NIO operate a type of data block, it’s bigger than a byte[maybe].

1. IO is block, NIO is unblock
   Traditional IO when a thread call read() or write(),
   thread need to waiting for data operation finished.
   During this time thread will be hang on? nothing it can be done.

To NIO when a thread send a request, but no response.
It’s a free, it can do any other tasks, until selector call.

Scene

NIO’s disadvantage is it need to check buffer space everytime.
Because it design for block, if block is not completed,
there is no meaning to operate data.

NIO fit for thousands short link and little data.
IO fit for few long link and big data.

Explain

Three guys got a task, to drink 50L water.
They all got a 100ml break. Have three water taps, it has no valve,
occasionally 100ml water flow out.

So these three poor guys have to wait until drink 500 cup of break.

Fantasy

But if they have a plumber to manage these water tap, thing wil be big different.

• First he can prepare three big bucket maybe 1L capacity each.
• Second he can install a valve in each tap.
• Third every guy told plumber their phone number,
  when bucket filled with water, plumber call them to come, and close valve.

Use this mechanism guy need to come and drink 5 times bucket.

Thinking

If the water tap’s water flow is very strong,
then this mechanism is not good.
Because plumber’s manage tap needs time, and it’s buckets needs space.
So in this condition, these guys waiting for break
be filled with water and drink is a better chooice.