在win系统搭建redis集群方法

我这边主要是搭建出来做测试用的,真实环境还是要使用linux。

环境需求:

Redis-win-3.2.100
Ruby-win-2.2.4-x64
Redis-3.2.2.gem(ruby驱动,需要对应redis的版本号)
Redis-trib.rb源码
1.安装Redis,并运行3个实例(Redis集群需要至少3个以上节点,低于3个无法创建);

2.使用redis-trib.rb工具来创建Redis集群,由于该文件是用ruby语言写的,所以需要安装Ruby开发环境,以及驱动redis-xxxx.gem。

1.下载并安装Redis

GitHub路径如下:https://github.com/MSOpenTech/redis/releases/

Redis提供msi和zip格式的下载文件,这里下载zip格式3.2.100版本

将下载到的Redis-win-3.2.100.zip解压即可,为了方便使用,建议放在盘符根目录下,并修改目录名为Redis,如:C:\Redis 或者D:\Redis。当然也可以放在桌面,只要你喜欢。

通过配置文件来启动3个不同的Redis实例,由于Redis默认端口为6379,所以这里使用了7000、7001、7002来运行3个Redis实例。

Spring框架简介

1. Spring框架的作用

轻量:Spring是轻量级的,基本的版本大小为2MB
控制反转:Spring通过控制反转实现了松散耦合,对象们给出它们的依赖,而不是创建或查找依赖的对象们。
面向切面的编程AOP:Spring支持面向切面的编程,并且把应用业务逻辑和系统服务分开。
容器:Spring包含并管理应用中对象的生命周期和配置
MVC框架: Spring-MVC
事务管理:Spring提供一个持续的事务管理接口,可以扩展到上至本地事务下至全局事务JTA
异常处理:Spring提供方便的API把具体技术相关的异常

tx:annotation-driven注解方式EnableTransactionManagement

替换方法是用@EnableTransactionManagement

以下是官方文档:

org.springframework.transaction.annotation
Annotation Type EnableTransactionManagement


@Target(value=TYPE)
@Retention(value=RUNTIME)
@Documented
@Import(value=TransactionManagementConfigurationSelector.class)
public @interface EnableTransactionManagement

Enables Spring’s annotation-driven transaction management capability, similar to the support found in Spring’s <tx:*> XML namespace. To be used on @Configuration classes as follows:

 @Configuration
 @EnableTransactionManagement
 public class AppConfig {
     @Bean
     public FooRepository fooRepository() {
         // configure and return a class having @Transactional methods
         return new JdbcFooRepository(dataSource());
     }

     @Bean
     public DataSource dataSource() {
         // configure and return the necessary JDBC DataSource
     }

     @Bean
     public PlatformTransactionManager txManager() {
         return new DataSourceTransactionManager(dataSource());
     }
 }

For reference, the example above can be compared to the following Spring XML configuration:

 <beans>
     <tx:annotation-driven/>
     <bean id="fooRepository" class="com.foo.JdbcFooRepository">
         <constructor-arg ref="dataSource"/>
     </bean>
     <bean id="dataSource" class="com.vendor.VendorDataSource"/>
     <bean id="transactionManager" class="org.sfwk...DataSourceTransactionManager">
         <constructor-arg ref="dataSource"/>
     </bean>
 </beans>
 

In both of the scenarios above, @EnableTransactionManagement and <tx:annotation-driven/> are responsible for registering the necessary Spring components that power annotation-driven transaction management, such as the TransactionInterceptor and the proxy- or AspectJ-based advice that weave the interceptor into the call stack when JdbcFooRepository‘s @Transactional methods are invoked.

A minor difference between the two examples lies in the naming of the PlatformTransactionManager bean: In the @Bean case, the name is “txManager” (per the name of the method); in the XML case, the name is“transactionManager”. The <tx:annotation-driven/> is hard-wired to look for a bean named “transactionManager” by default, however @EnableTransactionManagement is more flexible; it will fall back to a by-type lookup for anyPlatformTransactionManager bean in the container. Thus the name can be “txManager”, “transactionManager”, or “tm”: it simply does not matter.

For those that wish to establish a more direct relationship between @EnableTransactionManagement and the exact transaction manager bean to be used, the TransactionManagementConfigurer callback interface may be implemented – notice the implements clause and the @Override-annotated method below:

 @Configuration
 @EnableTransactionManagement
 public class AppConfig implements TransactionManagementConfigurer {
     @Bean
     public FooRepository fooRepository() {
         // configure and return a class having @Transactional methods
         return new JdbcFooRepository(dataSource());
     }

     @Bean
     public DataSource dataSource() {
         // configure and return the necessary JDBC DataSource
     }

     @Bean
     public PlatformTransactionManager txManager() {
         return new DataSourceTransactionManager(dataSource());
     }

     @Override
     public PlatformTransactionManager annotationDrivenTransactionManager() {
         return txManager();
     }
 }

This approach may be desirable simply because it is more explicit, or it may be necessary in order to distinguish between two PlatformTransactionManager beans present in the same container. As the name suggests, theannotationDrivenTransactionManager() will be the one used for processing @Transactional methods. See TransactionManagementConfigurer Javadoc for further details.

The mode() attribute controls how advice is applied; if the mode is AdviceMode.PROXY (the default), then the other attributes control the behavior of the proxying.

If the mode() is set to AdviceMode.ASPECTJ, then the proxyTargetClass() attribute is obsolete. Note also that in this case the spring-aspects module JAR must be present on the classpath.

Since:
3.1
Author:
Chris Beams
See Also:
TransactionManagementConfigurer, TransactionManagementConfigurationSelector, ProxyTransactionManagementConfiguration, AspectJTransactionManagementConfiguration

Optional Element Summary
 AdviceMode mode
Indicate how transactional advice should be applied.
 int order
Indicate the ordering of the execution of the transaction advisor when multiple advices are applied at a specific joinpoint.
 boolean proxyTargetClass
Indicate whether subclass-based (CGLIB) proxies are to be created (true) as opposed to standard Java interface-based proxies (false).

 

proxyTargetClass

public abstract boolean proxyTargetClass
Indicate whether subclass-based (CGLIB) proxies are to be created (true) as opposed to standard Java interface-based proxies (false). The default is false. Applicable only if mode() is set to AdviceMode.PROXY.Note that setting this attribute to true will affect all Spring-managed beans requiring proxying, not just those marked with @Transactional. For example, other beans marked with Spring’s @Async annotation will be upgraded to subclass proxying at the same time. This approach has no negative impact in practice unless one is explicitly expecting one type of proxy vs another, e.g. in tests.

Default:
false

mode

public abstract AdviceMode mode
Indicate how transactional advice should be applied. The default is AdviceMode.PROXY.
See Also:
AdviceMode
Default:
org.springframework.context.annotation.AdviceMode.PROXY

order

public abstract int order
Indicate the ordering of the execution of the transaction advisor when multiple advices are applied at a specific joinpoint. The default is Ordered.LOWEST_PRECEDENCE.
Default:
2147483647

 

https://docs.spring.io/spring/docs/4.0.x/javadoc-api/org/springframework/transaction/annotation/EnableTransactionManagement.html

更多文档:

https://docs.spring.io/spring/docs/4.0.x/javadoc-api/overview-summary.html

Java多线程,线程池ThreadPoolExecutor使用详解

ThreadPoolExecutor.class构造方法参数讲解

参数名 作用
corePoolSize 核心线程池大小
maximumPoolSize 最大线程池大小
keepAliveTime 线程池中超过corePoolSize数目的空闲线程最大存活时间;可以allowCoreThreadTimeOut(true)使得核心线程有效时间
TimeUnit keepAliveTime时间单位
workQueue 阻塞任务队列
threadFactory 新建线程工厂
RejectedExecutionHandler 当提交任务数超过maxmumPoolSize+workQueue之和时,任务会交给RejectedExecutionHandler来处理

1.当线程池小于corePoolSize时,新提交任务将创建一个新线程执行任务,即使此时线程池中存在空闲线程。
2.当线程池达到corePoolSize时,新提交任务将被放入workQueue中,等待线程池中任务调度执行
3.当workQueue已满,且maximumPoolSize>corePoolSize时,新提交任务会创建新线程执行任务
4.当提交任务数超过maximumPoolSize时,新提交任务由RejectedExecutionHandler处理
5.当线程池中超过corePoolSize线程,空闲时间达到keepAliveTime时,关闭空闲线程
6.当设置allowCoreThreadTimeOut(true)时,线程池中corePoolSize线程空闲时间达到keepAliveTime也将关闭

示例代码:

import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class testThreadPool {

   /**
    * 线程计数 
    */
   private static int count = 0;
   public static void main(String[] args) {
      
      /**
       * 通常会使用一个队列,进行排序
       */
      BlockingQueue<Runnable> queue = new ArrayBlockingQueue<>(25);
      
      /**
       * 如果使用无界的队列,当出现攻击时,容易导致内存泄漏。
       */
      //LinkedBlockingDeque<Runnable> queue = new LinkedBlockingDeque<>();
      
      /**
       * 线程池
       */
      ThreadPoolExecutor pool = new ThreadPoolExecutor(5, 10, 30, TimeUnit.MINUTES, queue);
      
      /**
       * 重写Handler抛出来的线程重新加进去
       */
      pool.setRejectedExecutionHandler(new RejectedExecutionHandler() {
         
         @Override
         public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
            // TODO Auto-generated method stub
            while (true) {
               if (queue.size() < 24) {
                  System.out.println("++++++++++++++Runnable");
                  executor.execute(r);
                  break;
               }
            }
         }
      });

      /**
       * 定义一个定时器,每2秒创建10个进程
       */
      Timer timer = new Timer();
      timer.schedule(new TimerTask() {
         
         @Override
         public void run() {
            // TODO Auto-generated method stub
            
            for(int i = 0;i<10;i++) {
               count++;
               doWork work = new doWork();
               pool.execute(work);
               
            }
            
            if (count >= 200) {
               timer.cancel();
            }
         }
      }, 0,2000);
      
      /**
       * 定时检查线程的状况
       */
      new Timer().schedule(new TimerTask() {
         @Override
         public void run() {
            // TODO Auto-generated method stub
            System.err.println(">>当前队列:"+queue.size());
            System.err.println(">>线程数:"+count);
            System.err.println(">>当前ActiveCount:"+pool.getActiveCount());
            System.err.println(">>当前TaskCount:"+pool.getTaskCount());
            System.err.println(">>当前PoolSize:"+pool.getPoolSize());
         }
      }, 1000,1000);
      
   }
}

class doWork implements Runnable{
   

   @Override
   public void run() {
      // TODO Auto-generated method stub
      
      try {
         /**
          * 模拟真实环境,睡眠3秒
          */
         Thread.sleep(3000);
         System.out.println("线程id:"+Thread.currentThread().getId()+" 线程名称:"+Thread.currentThread().getName());
      } catch (InterruptedException e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }
      
   }

}

 

API文档:

http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html

myBatis数据库自动断开的解决办法

使用Spring+SpringMVC+myBatis+Mysql开发一个应用服务端,由于在测试阶段没有上线,所以没有人会链接服务器。

放假回来才发现数据库被关闭了,而且要重启tomcat才有效,都不知道myBatis为什么会没有自动重连数据库的功能。

异常信息:

23-Sep-2017 09:43:39.880 SEVERE [http-nio-8081-exec-93] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [dispatcher] in context with path [/project] threw exception [Request processing failed; nested exception is org.springframework.dao.DataAccessResourceFailureException:
### Error querying database. Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
### The error may exist in file [/www/apache-tomcat-8081/webapps/project/WEB-INF/classes/com/test/system/mapper/xml/cateMapper.xml]
### The error may involve com.test.system.mapper.SdGoodsCateMapper.getCateByGroup
### The error occurred while executing a query
### SQL: SELECT b.* FROM …..;
### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
; SQL []; No operations allowed after connection closed.; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.] with root cause
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3728)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2509)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2490)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2448)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1381)
at com.mysql.jdbc.DatabaseMetaData.getUserName(DatabaseMetaData.java:6274)
at org.apache.commons.dbcp.DelegatingConnection.toString(DelegatingConnection.java:123)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.toString(PoolingDataSource.java:355)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:87)
at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:68)
at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:315)
at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:75)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:61)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:303)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:154)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:102)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:82)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:120)
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:113)
at sun.reflect.GeneratedMethodAccessor2495.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:408)
at com.sun.proxy.$Proxy408.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:206)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:122)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:64)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
at com.sun.proxy.$Proxy434.getCateByGroup(Unknown Source)
at com.test.system.service.HomeServiceImpl.getHomeAlbumByHot(HomeServiceImpl.java:95)
at com.test.app.controller.HomeController.getAlbumByHot(HomeController.java:60)
at sun.reflect.GeneratedMethodAccessor2778.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:222)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:814)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:737)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:969)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:860)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:845)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1533)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1489)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

原因分析:

查看了Mysql的文档,以及Connector/J的文档以及在线说明发现,出现这种异常的原因是:

Mysql服务器默认的“wait_timeout”是8小时,也就是说一个connection空闲超过8个小时,Mysql将自动断开该connection。这就是问题的所在,在C3P0 pools中的connections如果空闲超过8小时,Mysql将其断开,而C3P0并不知道该connection已经失效,如果这时有Client请求connection,C3P0将该失效的Connection提供给Client,将会造成上面的异常。

解决方案:

加上如下配置:

//SQL查询,用来验证从连接池取出的连接,在将连接返回给调用者之前.如果指定,则查询必须是一个SQL SELECT并且必须返回至少一行记录

validationQuery = select 1

//指明连接是否被空闲连接回收器(如果有)进行检验.如果检测失败,则连接将被从池中去除.

testWhileIdle = true

//在空闲连接回收器线程运行期间休眠的时间值,以毫秒为单位.如果设置为非正数,则不运行空闲连接回收器线程

timeBetweenEvictionRunsMillis = 3600000

//1000 * 60 * 30  连接在池中保持空闲而不被空闲连接回收器线程,(如果有)回收的最小时间值,单位毫秒

minEvictableIdleTimeMillis = 18000000

//指明是否在从池中取出连接前进行检验,如果检验失败,则从池中去除连接并尝试取出另一个.

testOnBorrow = true

 

validationQuery参考:

validationQuery是用来验证数据库连接的查询语句,这个查询语句必须是至少返回一条数据的SELECT语句。每种数据库都有各自的验证语句,下表中收集了几种常见数据库的validationQuery。

DataBase validationQuery
hsqldb select 1 from INFORMATION_SCHEMA.SYSTEM_USERS
Oracle select 1 from dual
DB2 select 1 from sysibm.sysdummy1
MySql select 1
Microsoft SqlServer select1
postgresql select version()
ingres select 1
derby values 1
H2 select 1

 

BasicDataSource Configuration Parameters:

http://commons.apache.org/proper/commons-dbcp/configuration.html

 

1272829303132
 
Copyright © 2008-2021 lanxinbase.com Rights Reserved. | 粤ICP备14086738号-3 |