黑帽联盟

 找回密码
 会员注册
查看: 2163|回复: 0
打印 上一主题 下一主题

[集群服务] (nginx|apache 反向代理)+Tomcat+Memcached实现会话共享

[复制链接]

295

主题

18

听众

965

积分

管理员

Rank: 9Rank: 9Rank: 9

  • TA的每日心情
    难过
    前天 00:16
  • 签到天数: 370 天

    [LV.9]以坛为家II

    前一篇文章我们介绍了session会话共享,在负载均衡主机上进行设置sticky相关设置以实现session会话共享;以及在后端tomcat主机设置session cluster以实现session会话共享

    相关链接nginx、apache反向代理用户请求至后端Tomcat 集群以及session绑定,session共享


    这篇文章我们来介绍一下通过memcached来缓存session,以实现会话共享

    首先反向代理(负载均衡)配置这里就不介绍了,跳过,不会的,点击上面的相关链接,自习一下

    环境:
    nginx|apache反向代理节点:172.31.225.246 (centos6)
    tomcat节点:172.31.225.243(TomcatA),172.31.225.244(TomcatB) (都是centos7.3)
    memcached节点(一主一备实现高可用):172.31.225.243,172.31.225.244 (由于自己机器配置原因,这里把memcached和tomcat这两个服务放在一块)

    实验拓扑图:(来源网络)

    212121.jpg

    在两台tomcat主机上安装memcahched
    yum install memcached -y

    启动memcached服务
    systemctl start memcached

    检查启动成功与否:
    ss -tnl | grep 11211

    2222.png

    3333.png

    接着配置tomcat,这里还是以 https://bbs.cnblackhat.com/thread-2776-1-1.html 版本为主,tomcat8.5版本,下载地址:http://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-8/v8.5.46/bin/apache-tomcat-8.5.46.tar.gz

    tar xf apache-tomcat-8.5.46.tar.gz -C /usr/local/
    ln -sv apache-tomcat-8.5.46 tomcat
    vim /etc/profile.d/tomcat.sh #内容如下
    export CATALINA_HOME=/usr/local/tomcat
    export PATH=$CATALINA_HOME/bin:$PATH

    最后加载环境变量,使其生效:
    . /etc/profile.d/tomcat.sh

    由于centos7默认自带openjdk,这里就不配置JDK环境变量了

    最关键的就是tomcat的配置了
    实现会话保持我们用到的是memcached-session-manager,而需要下几个类库,我已经下载好了。这个是项目官方地址,可以到这里进行下载http://code.google.com/p/memcached-session-manager/

    由于该网址被墙,我是在其它地方下载的,相关文件我已打包,文件截图:
    1212.png

    文件下载 memcached-session-manager.zip (990.7 KB, 下载次数: 0)
    文件下载相关链接:http://repo1.maven.org/maven2/de/javakaffee/msm/

    将这几个类库放到tomcat的lib库中
    cp * /usr/local/tomcat/lib/

    进去看下,确保复制成功

    好了下面就开始修改配置文件 /usr/local/tomcat/conf/server.xml
    分别在两个tomcat上的host上定义一个用于测试的context容器,并在其中创建一个会话管理器,如下所示:(直接贴出两台tomcat配置文件server.xml)
    172.31.225.243(TomcatA)server.xml配置文件
    <?xml version="1.0" encoding="UTF-8"?>
    <!--
      Licensed to the Apache Software Foundation (ASF) under one or more
      contributor license agreements.  See the NOTICE file distributed with
      this work for additional information regarding copyright ownership.
      The ASF licenses this file to You under the Apache License, Version 2.0
      (the "License"); you may not use this file except in compliance with
      the License.  You may obtain a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License.
    -->
    <!-- Note:  A "Server" is not itself a "Container", so you may not
         define subcomponents such as "Valves" at this level.
         Documentation at /docs/config/server.html
    -->
    <Server port="8005" shutdown="SHUTDOWN">
      <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
      <!-- Security listener. Documentation at /docs/config/listeners.html
      <Listener className="org.apache.catalina.security.SecurityListener" />
      -->
      <!--APR library loader. Documentation at /docs/apr.html -->
      <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
      <!-- Prevent memory leaks due to use of particular java/javax APIs-->
      <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
      <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
      <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

      <!-- Global JNDI resources
           Documentation at /docs/jndi-resources-howto.html
      -->
      <GlobalNamingResources>
        <!-- Editable user database that can also be used by
             UserDatabaseRealm to authenticate users
        -->
        <Resource name="UserDatabase" auth="Container"
                  type="org.apache.catalina.UserDatabase"
                  description="User database that can be updated and saved"
                  factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
                  pathname="conf/tomcat-users.xml" />
      </GlobalNamingResources>

      <!-- A "Service" is a collection of one or more "Connectors" that share
           a single "Container" Note:  A "Service" is not itself a "Container",
           so you may not define subcomponents such as "Valves" at this level.
           Documentation at /docs/config/service.html
       -->
      <Service name="Catalina">

        <!--The connectors can use a shared executor, you can define one or more named thread pools-->
        <!--
        <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
            maxThreads="150" minSpareThreads="4"/>
        -->


        <!-- A "Connector" represents an endpoint by which requests are received
             and responses are returned. Documentation at :
             Java HTTP Connector: /docs/config/http.html
             Java AJP  Connector: /docs/config/ajp.html
             APR (HTTP/AJP) Connector: /docs/apr.html
             Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
        -->
        <Connector port="8080" protocol="HTTP/1.1"
                   connectionTimeout="20000"
                   redirectPort="8443" />
        <!-- A "Connector" using the shared thread pool-->
        <!--
        <Connector executor="tomcatThreadPool"
                   port="8080" protocol="HTTP/1.1"
                   connectionTimeout="20000"
                   redirectPort="8443" />
        -->
        <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443
             This connector uses the NIO implementation. The default
             SSLImplementation will depend on the presence of the APR/native
             library and the useOpenSSL attribute of the
             AprLifecycleListener.
             Either JSSE or OpenSSL style configuration may be used regardless of
             the SSLImplementation selected. JSSE style configuration is used below.
        -->
        <!--
        <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
                   maxThreads="150" SSLEnabled="true">
            <SSLHostConfig>
                <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
                             type="RSA" />
            </SSLHostConfig>
        </Connector>
        -->
        <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
             This connector uses the APR/native implementation which always uses
             OpenSSL for TLS.
             Either JSSE or OpenSSL style configuration may be used. OpenSSL style
             configuration is used below.
        -->
        <!--
        <Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
                   maxThreads="150" SSLEnabled="true" >
            <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
            <SSLHostConfig>
                <Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
                             certificateFile="conf/localhost-rsa-cert.pem"
                             certificateChainFile="conf/localhost-rsa-chain.pem"
                             type="RSA" />
            </SSLHostConfig>
        </Connector>
        -->

        <!-- Define an AJP 1.3 Connector on port 8009 -->
        <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


        <!-- An Engine represents the entry point (within Catalina) that processes
             every request.  The Engine implementation for Tomcat stand alone
             analyzes the HTTP headers included with the request, and passes them
             on to the appropriate Host (virtual host).
             Documentation at /docs/config/engine.html -->

        <!-- You should set jvmRoute to support load-balancing via AJP ie :
        <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
        -->
        <Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatA">

          <!--For clustering, please take a look at documentation at:
              /docs/cluster-howto.html  (simple how to)
              /docs/config/cluster.html (reference documentation) -->
          <!--
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
          -->

          <!-- Use the LockOutRealm to prevent attempts to guess user passwords
               via a brute-force attack -->
          <Realm className="org.apache.catalina.realm.LockOutRealm">
            <!-- This Realm uses the UserDatabase configured in the global JNDI
                 resources under the key "UserDatabase".  Any edits
                 that are performed against this UserDatabase are immediately
                 available for use by the Realm.  -->
            <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                   resourceName="UserDatabase"/>
          </Realm>

          <Host name="localhost"  appBase="webapps"
                unpackWARs="true" autoDeploy="true">
            <Context path="/myapp" docBase="myapp" reloadable="true">
                    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
                            memcachedNodes="n1:172.31.225.243:11211,n2:172.31.225.244:11211"
                            failoverNodes="n1"
                            requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
                            transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"/>
            </Context>
            <!-- SingleSignOn valve, share authentication between web applications
                 Documentation at: /docs/config/valve.html -->
            <!--
            <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
            -->

            <!-- Access log processes all example.
                 Documentation at: /docs/config/valve.html
                 Note: The pattern used is equivalent to using pattern="common" -->
            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                   prefix="localhost_access_log" suffix=".txt"
                   pattern="%h %l %u %t "%r" %s %b" />

          </Host>
        </Engine>
      </Service>
    </Server>
    172.31.225.244(TomcatB)server.xml配置文件
    <?xml version="1.0" encoding="UTF-8"?>
    <!--
      Licensed to the Apache Software Foundation (ASF) under one or more
      contributor license agreements.  See the NOTICE file distributed with
      this work for additional information regarding copyright ownership.
      The ASF licenses this file to You under the Apache License, Version 2.0
      (the "License"); you may not use this file except in compliance with
      the License.  You may obtain a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License.
    -->
    <!-- Note:  A "Server" is not itself a "Container", so you may not
         define subcomponents such as "Valves" at this level.
         Documentation at /docs/config/server.html
    -->
    <Server port="8005" shutdown="SHUTDOWN">
      <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
      <!-- Security listener. Documentation at /docs/config/listeners.html
      <Listener className="org.apache.catalina.security.SecurityListener" />
      -->
      <!--APR library loader. Documentation at /docs/apr.html -->
      <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
      <!-- Prevent memory leaks due to use of particular java/javax APIs-->
      <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
      <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
      <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

      <!-- Global JNDI resources
           Documentation at /docs/jndi-resources-howto.html
      -->
      <GlobalNamingResources>
        <!-- Editable user database that can also be used by
             UserDatabaseRealm to authenticate users
        -->
        <Resource name="UserDatabase" auth="Container"
                  type="org.apache.catalina.UserDatabase"
                  description="User database that can be updated and saved"
                  factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
                  pathname="conf/tomcat-users.xml" />
      </GlobalNamingResources>

      <!-- A "Service" is a collection of one or more "Connectors" that share
           a single "Container" Note:  A "Service" is not itself a "Container",
           so you may not define subcomponents such as "Valves" at this level.
           Documentation at /docs/config/service.html
       -->
      <Service name="Catalina">

        <!--The connectors can use a shared executor, you can define one or more named thread pools-->
        <!--
        <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
            maxThreads="150" minSpareThreads="4"/>
        -->


        <!-- A "Connector" represents an endpoint by which requests are received
             and responses are returned. Documentation at :
             Java HTTP Connector: /docs/config/http.html
             Java AJP  Connector: /docs/config/ajp.html
             APR (HTTP/AJP) Connector: /docs/apr.html
             Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
        -->
        <Connector port="8080" protocol="HTTP/1.1"
                   connectionTimeout="20000"
                   redirectPort="8443" />
        <!-- A "Connector" using the shared thread pool-->
        <!--
        <Connector executor="tomcatThreadPool"
                   port="8080" protocol="HTTP/1.1"
                   connectionTimeout="20000"
                   redirectPort="8443" />
        -->
        <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443
             This connector uses the NIO implementation. The default
             SSLImplementation will depend on the presence of the APR/native
             library and the useOpenSSL attribute of the
             AprLifecycleListener.
             Either JSSE or OpenSSL style configuration may be used regardless of
             the SSLImplementation selected. JSSE style configuration is used below.
        -->
        <!--
        <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
                   maxThreads="150" SSLEnabled="true">
            <SSLHostConfig>
                <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
                             type="RSA" />
            </SSLHostConfig>
        </Connector>
        -->
        <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
             This connector uses the APR/native implementation which always uses
             OpenSSL for TLS.
             Either JSSE or OpenSSL style configuration may be used. OpenSSL style
             configuration is used below.
        -->
        <!--
        <Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
                   maxThreads="150" SSLEnabled="true" >
            <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
            <SSLHostConfig>
                <Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
                             certificateFile="conf/localhost-rsa-cert.pem"
                             certificateChainFile="conf/localhost-rsa-chain.pem"
                             type="RSA" />
            </SSLHostConfig>
        </Connector>
        -->

        <!-- Define an AJP 1.3 Connector on port 8009 -->
        <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


        <!-- An Engine represents the entry point (within Catalina) that processes
             every request.  The Engine implementation for Tomcat stand alone
             analyzes the HTTP headers included with the request, and passes them
             on to the appropriate Host (virtual host).
             Documentation at /docs/config/engine.html -->

        <!-- You should set jvmRoute to support load-balancing via AJP ie :
        <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
        -->
        <Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatB">

          <!--For clustering, please take a look at documentation at:
              /docs/cluster-howto.html  (simple how to)
              /docs/config/cluster.html (reference documentation) -->
          <!--
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
          -->

          <!-- Use the LockOutRealm to prevent attempts to guess user passwords
               via a brute-force attack -->
          <Realm className="org.apache.catalina.realm.LockOutRealm">
            <!-- This Realm uses the UserDatabase configured in the global JNDI
                 resources under the key "UserDatabase".  Any edits
                 that are performed against this UserDatabase are immediately
                 available for use by the Realm.  -->
            <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                   resourceName="UserDatabase"/>
          </Realm>

          <Host name="localhost"  appBase="webapps"
                unpackWARs="true" autoDeploy="true">
            <Context path="/myapp" docBase="myapp" reloadable="true">
                    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
                            memcachedNodes="n1:172.31.225.243:11211,n2:172.31.225.244:11211"
                            failoverNodes="n1"
                            requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
                            transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"/>
            </Context>
            <!-- SingleSignOn valve, share authentication between web applications
                 Documentation at: /docs/config/valve.html -->
            <!--
            <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
            -->

            <!-- Access log processes all example.
                 Documentation at: /docs/config/valve.html
                 Note: The pattern used is equivalent to using pattern="common" -->
            <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
                   prefix="localhost_access_log" suffix=".txt"
                   pattern="%h %l %u %t "%r" %s %b" />

          </Host>
        </Engine>
      </Service>
    </Server>
    主要是在Host标签内部,添加这么一段:
    <Context path="/myapp" docBase="myapp" reloadable="true">
                    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
                            memcachedNodes="n1:172.31.225.243:11211,n2:172.31.225.244:11211"
                            failoverNodes="n1"
                            requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
                            transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"/>
            </Context>
    提供测试相应的文件夹及测试文件
    tomcatA:
    # mkdir -pv /usr/local/tomcat/webapps/myapp/WEB-INF/{classes,lib}
      mkdir: created directory `/usr/local/tomcat/webapps/myapp'   
      mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF'   
      mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF/classes'   
      mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF/lib'
    # vim /usr/local/tomcat/webapps/myapp/index.jsp
    <%@ page language="java" %>
    <html>
      <head><title>TomcatA_172.31.225.243</title></head>
      <body>
        <h1><font color="red">tomcatA.cnblackhat.com</font></h1>
        <table align="centre" border="1">
          <tr>
            <td>Session ID</td>
        <% session.setAttribute("cnblackhat.com","cnblackhat.com"); %>
            <td><%= session.getId() %></td>
          </tr>
          <tr>
            <td>Created on</td>
            <td><%= session.getCreationTime() %></td>
         </tr>
        </table>
      </body>
    </html>
    复制web.xml文件
    # cp /usr/local/tomcat/conf/web.xml /usr/local/tomcat/webapps/test/WEB-INF/
    tomcatB:
    # mkdir -pv /usr/local/tomcat/webapps/myapp/WEB-INF/{classes,lib}
    mkdir: created directory `/usr/local/tomcat/webapps/myapp'
    mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF'
    mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF/classes'
    mkdir: created directory `/usr/local/tomcat/webapps/myapp/WEB-INF/lib'
    # vim /usr/local/tomcat/webapps/myapp/index.jsp
    <%@ page language="java" %>
    <html>
    <head><title>TomcatB_172.31.225.244</title></head>
      <body>
        <h1><font color="red">tomcatB.cnblackhat.com </font></h1>
        <table align="centre" border="1">
          <tr>
            <td>Session ID</td>
        <% session.setAttribute("cnblackhat.com","cnblackhat.com"); %>
            <td><%= session.getId() %></td>
          </tr>
          <tr>
            <td>Created on</td>
            <td><%= session.getCreationTime() %></td>
         </tr>
        </table>
      </body>
    </html>
    复制web.xml文件
    # cp /usr/local/tomcat/conf/web.xml /usr/local/tomcat/webapps/test/WEB-INF/
    好了下面我们就可以开始测试了。
    111111.png

    222222.png


    动态图展示:
    121212.gif


    OK,sessionID没变,会话成功保持。

    172.31.225.244节点上查询memcached缓存,截图如下:session缓存成功:
    1212.png

    测试memcached高可用,此时我们把172.31.225.244这台主机上的memcached服务给关闭了,在浏览器上在访问172.31.225.246/myapp,结果依然可以访问,高可用实现成功,如图:
    2122.png

    2121.png

    而此时的session缓存保存在172.31.225.243节点上,如下图:
    12111.png


    帖子永久地址: 

    黑帽联盟 - 论坛版权1、本主题所有言论和图片纯属会员个人意见,与本论坛立场无关
    2、本站所有主题由该帖子作者发表,该帖子作者与黑帽联盟享有帖子相关版权
    3、其他单位或个人使用、转载或引用本文时必须同时征得该帖子作者和黑帽联盟的同意
    4、帖子作者须承担一切因本文发表而直接或间接导致的民事或刑事法律责任
    5、本帖部分内容转载自其它媒体,但并不代表本站赞同其观点和对其真实性负责
    6、如本帖侵犯到任何版权问题,请立即告知本站,本站将及时予与删除并致以最深的歉意
    7、黑帽联盟管理员和版主有权不事先通知发贴者而删除本文

    您需要登录后才可以回帖 登录 | 会员注册

    发布主题 !fastreply! 收藏帖子 返回列表 搜索
    回顶部