HTTP的GET请求RUL保留字处理方式（多个解决方案）
程序员文章站
2022-07-16 14:01:25
...
背景：
最近花了一天在处理一个生产环境问题，
客户端（发送数据）：通过HTTP的GET请求，传输参数中带有“+”加号。
服务端（接收数据）：“+”加号变为空格。
因为是签名数据，导致服务端验证签名不通过，算比较严重的问题。
解决问题示例（多个解决方案）：
示例1（请求url的参数采用直接拼装的方式）（失败）：
package com.qhfax.test;
import com.qhfax.common.util.HttpClientUtil;
public class HttpGetTest1 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
//请求参数
String queryString = "?sign="+sign;
//URI编码处理
String getUrl = serviceUrl + queryString;
String response = HttpClientUtil.get(getUrl);
System.out.println(response);
}
}
服务端接收的的值为：abcde fghij
服务端返回：false
示例2（使用URIUtil.encodeQuery方法对请求参数进行编码）（失败）
package com.qhfax.test;
import org.apache.commons.httpclient.util.URIUtil;
import com.qhfax.common.util.HttpClientUtil;
public class HttpGetTest2 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
//请求参数
String queryString = "?sign="+sign;
//URI编码处理
String getUrl = serviceUrl + URIUtil.encodeQuery(queryString);
String response = HttpClientUtil.get(getUrl);
System.out.println(response);
}
}
服务端接收的的值为：abcde fghij
服务端返回：false
示例3（对URL保留字符进行ASCII码转换，把“+”替换为“%2B”）（成功）
package com.qhfax.test;
import com.qhfax.common.util.HttpClientUtil;
public class HttpGetTest3 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//把“+”替换为“%2B”
sign = sign.replaceAll("\\+", "%2B");
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
//请求参数
String queryString = "?sign="+sign;
//URI编码处理
String getUrl = serviceUrl + queryString;
String response = HttpClientUtil.get(getUrl);
System.out.println(response);
}
}
服务端接收的的值为：abcde+fghij
服务端返回：true
示例4（基于示例3，使用java自带的URLEncoder.encode方法对参数进行编码）（成功）
package com.qhfax.test;
import com.qhfax.common.util.HttpClientUtil;
public class HttpGetTest4 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//使用java自带的URLEncoder.encode方法对参数进行编码
sign = java.net.URLEncoder.encode(sign);
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
//请求参数
String queryString = "?sign="+sign;
//URI编码处理
String getUrl = serviceUrl + queryString;
String response = HttpClientUtil.get(getUrl);
System.out.println(response);
}
}
服务端接收的的值为：abcde+fghij
服务端返回：true
示例5（基于示例3、4，使用httpClient包中的UrlEncodedFormEntity类，详见上面的HttpClientUtil工具类）（最终版本）（成功）
package com.qhfax.test;
import java.util.HashMap;
import java.util.Map;
import com.qhfax.common.util.HttpClientUtil;
public class HttpGetTest5 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
//请求参数
Map<String, String> params = new HashMap<String, String>();
params.put("sign", sign);
String response = HttpClientUtil.get(serviceUrl, params);
System.out.println(response);
}
}
服务端接收的的值为：abcde+fghij
服务端返回：true
示例6（换一种方式，也是Httpclientjar包中的类，使用Request.Get）（成功）
package com.qhfax.test;
import org.apache.http.client.fluent.Request;
import org.apache.http.client.utils.URIBuilder;
public class HttpGetTest6 {
public static void main(String[] args) throws Exception {
//签名
String sign = "abcde+fghij";
//请求的服务地址
String serviceUrl = "http://localhost:8080/qhfaxWeb/httpGetTest/paramTest";//请求路径
String result = Request.Get(
new URIBuilder(serviceUrl)
.addParameter("sign", sign)
.build())
.connectTimeout(5000)
.socketTimeout(5000).execute()
.returnContent().asString();
System.out.println(result);
}
}
服务端接收的的值为：abcde+fghij
服务端返回：true
总结：
1、找到问题：
根据示例1中问题，HTTP的GET请求URL中有包括“! * ' ( ) ; : @ & =+ $ , / ? # [ ]”的保留字符，需要对它们进行转码，这是解决问题思路的第一步，找到原因。
2、验证解决方案：
根据示例3中的处理方式，将把“+”替换为“%2B”，验证处理方法是正确的。这是解决问题思路的第二步，尝试方案，验证能否解决问题。
3、完善解决方案：
示例4、5、6是基于示例3的完善，利用现有的工具类，更好更全面的处理问题。这是解决问题思路的第三步，完善解决方案。
（备注：示例2中的URIUtil.encodeQuery是有编码效果的，只是好像对“+”不会编码，这个点在我处理时有点误导了我的思路）
演示准备前需准备的代码(注：SpringMVC框架环境需自己搭建，httpclient的jar版本为4.4.1)：
客户端：
HttpClientUtil工具类代码如下：
package com.qhfax.common.util;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.SocketTimeoutException;
import java.nio.charset.Charset;
import java.security.GeneralSecurityException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocket;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.http.Consts;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.ParseException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.config.RequestConfig.Builder;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContextBuilder;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.conn.ssl.X509HostnameVerifier;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* @Description: 使用httpclient4.0以上组件
* @author : huangaming
* @date : 2017年4月6日 上午11:49:48
*/
@SuppressWarnings("deprecation")
public class HttpClientUtil {
private static Logger logger = LoggerFactory.getLogger(HttpClientUtil.class);
public static String postJsonString(String uri, String jsonStr) {
String result = "";
Charset charset = Charset.forName("UTF-8");
// 实例化http客户端
HttpClient httpClient = HttpClientBuilder.create().build();
HttpPost post = null;
try {
post = new HttpPost(uri);
StringEntity stringEntity = new StringEntity(jsonStr, ContentType.create("application/json", charset));
// 实例化post提交方式
post.addHeader(HTTP.CONTENT_TYPE, ContentType.APPLICATION_JSON.toString());
// 将参数加入post请求体中
post.setEntity(stringEntity);
// 执行post请求并得到返回对象 [ 到这一步我们的请求就开始了 ]
HttpResponse resp = httpClient.execute(post);
// 解析返回请求结果
HttpEntity entity = resp.getEntity();
result = IOUtils.toString(entity.getContent(), charset);
logger.info("[postJsonString response:{}]", result);
// 输出结果
} catch (Exception exception) {
logger.error("postJsonString exception", exception);
} finally {
if (post != null) {
post.releaseConnection();
}
}
return result;
}
public static final int connTimeout=10000;//连接超时参数
public static final int readTimeout=10000;//读取超时参数
public static final String charset="UTF-8";//字符编码
private static HttpClient client = null;
static {
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(128);
cm.setDefaultMaxPerRoute(128);
client = HttpClients.custom().setConnectionManager(cm).build();
}
public static String postParameters(String url, String parameterStr) throws ConnectTimeoutException, SocketTimeoutException, Exception{
return post(url,parameterStr,"application/x-www-form-urlencoded",charset,connTimeout,readTimeout);
}
public static String postParameters(String url, String parameterStr,String charset, Integer connTimeout, Integer readTimeout) throws ConnectTimeoutException, SocketTimeoutException, Exception{
return post(url,parameterStr,"application/x-www-form-urlencoded",charset,connTimeout,readTimeout);
}
public static String postParameters(String url, Map<String, String> params) throws ConnectTimeoutException,
SocketTimeoutException, Exception {
return postForm(url, params, null, connTimeout, readTimeout);
}
public static String postParameters(String url, Map<String, String> params, Integer connTimeout,Integer readTimeout) throws ConnectTimeoutException,
SocketTimeoutException, Exception {
return postForm(url, params, null, connTimeout, readTimeout);
}
public static String get(String url) throws Exception {
return get(url, charset, connTimeout, readTimeout);
}
public static String get(String url, String charset) throws Exception {
return get(url, charset, connTimeout, readTimeout);
}
public static String get(String url, Map<String, String> params) throws Exception {
return get(url, params, charset, connTimeout, readTimeout);
}
/**
* 发送一个 Post 请求, 使用指定的字符集编码.
*
* @param url
* @param body RequestBody
* @param mimeType 例如 application/xml "application/x-www-form-urlencoded" a=1&b=2&c=3
* @param charset 编码
* @param connTimeout 建立链接超时时间,毫秒.
* @param readTimeout 响应超时时间,毫秒.
* @return ResponseBody, 使用指定的字符集编码.
* @throws ConnectTimeoutException 建立链接超时异常
* @throws SocketTimeoutException 响应超时
* @throws Exception
*/
public static String post(String url, String body, String mimeType,String charset, Integer connTimeout, Integer readTimeout)
throws ConnectTimeoutException, SocketTimeoutException, Exception {
HttpClient client = null;
HttpPost post = new HttpPost(url);
String result = "";
try {
if (StringUtils.isNotBlank(body)) {
HttpEntity entity = new StringEntity(body, ContentType.create(mimeType, charset));
post.setEntity(entity);
}
// 设置参数
Builder customReqConf = RequestConfig.custom();
if (connTimeout != null) {
customReqConf.setConnectTimeout(connTimeout);
}
if (readTimeout != null) {
customReqConf.setSocketTimeout(readTimeout);
}
post.setConfig(customReqConf.build());
HttpResponse res;
if (url.startsWith("https")) {
// 执行 Https 请求.
client = createSSLInsecureClient();
res = client.execute(post);
} else {
// 执行 Http 请求.
client = HttpClientUtil.client;
res = client.execute(post);
}
result = IOUtils.toString(res.getEntity().getContent(), charset);
} finally {
post.releaseConnection();
if (url.startsWith("https") && client != null&& client instanceof CloseableHttpClient) {
((CloseableHttpClient) client).close();
}
}
return result;
}
/**
* 提交form表单
*
* @param url
* @param params
* @param connTimeout
* @param readTimeout
* @return
* @throws ConnectTimeoutException
* @throws SocketTimeoutException
* @throws Exception
*/
public static String postForm(String url, Map<String, String> params, Map<String, String> headers, Integer connTimeout,Integer readTimeout) throws ConnectTimeoutException,
SocketTimeoutException, Exception {
HttpClient client = null;
HttpPost post = new HttpPost(url);
try {
if (params != null && !params.isEmpty()) {
List<NameValuePair> formParams = new ArrayList<org.apache.http.NameValuePair>();
Set<Entry<String, String>> entrySet = params.entrySet();
for (Entry<String, String> entry : entrySet) {
formParams.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
}
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams, Consts.UTF_8);
post.setEntity(entity);
}
if (headers != null && !headers.isEmpty()) {
for (Entry<String, String> entry : headers.entrySet()) {
post.addHeader(entry.getKey(), entry.getValue());
}
}
// 设置参数
Builder customReqConf = RequestConfig.custom();
if (connTimeout != null) {
customReqConf.setConnectTimeout(connTimeout);
}
if (readTimeout != null) {
customReqConf.setSocketTimeout(readTimeout);
}
post.setConfig(customReqConf.build());
HttpResponse res = null;
if (url.startsWith("https")) {
// 执行 Https 请求.
client = createSSLInsecureClient();
res = client.execute(post);
} else {
// 执行 Http 请求.
client = HttpClientUtil.client;
res = client.execute(post);
}
return IOUtils.toString(res.getEntity().getContent(), "UTF-8");
} finally {
post.releaseConnection();
if (url.startsWith("https") && client != null
&& client instanceof CloseableHttpClient) {
((CloseableHttpClient) client).close();
}
}
}
/**
* 发送一个 GET 请求
*
* @param url
* @param charset
* @param connTimeout 建立链接超时时间,毫秒.
* @param readTimeout 响应超时时间,毫秒.
* @return
* @throws ConnectTimeoutException 建立链接超时
* @throws SocketTimeoutException 响应超时
* @throws Exception
*/
public static String get(String url, String charset, Integer connTimeout,Integer readTimeout)
throws ConnectTimeoutException,SocketTimeoutException, Exception {
HttpClient client = null;
HttpGet get = new HttpGet(url);
String result = "";
try {
// 设置参数
Builder customReqConf = RequestConfig.custom();
if (connTimeout != null) {
customReqConf.setConnectTimeout(connTimeout);
}
if (readTimeout != null) {
customReqConf.setSocketTimeout(readTimeout);
}
get.setConfig(customReqConf.build());
HttpResponse res = null;
if (url.startsWith("https")) {
logger.info("httpClientUtil|get|执行https的get请求|开始");
// 执行 Https 请求.
client = createSSLInsecureClient();
res = client.execute(get);
logger.info("httpClientUtil|get|执行https的get请求|结束");
} else {
logger.info("httpClientUtil|get|执行http的get请求|开始");
// 执行 Http 请求.
client = HttpClientUtil.client;
res = client.execute(get);
logger.info("httpClientUtil|get|执行http的get请求|结束");
}
result = IOUtils.toString(res.getEntity().getContent(), charset);
} finally {
get.releaseConnection();
if (url.startsWith("https") && client != null && client instanceof CloseableHttpClient) {
((CloseableHttpClient) client).close();
}
}
return result;
}
/**
* 从 response 里获取 charset
*
* @param ressponse
* @return
*/
@SuppressWarnings("unused")
private static String getCharsetFromResponse(HttpResponse ressponse) {
// Content-Type:text/html; charset=GBK
if (ressponse.getEntity() != null && ressponse.getEntity().getContentType() != null && ressponse.getEntity().getContentType().getValue() != null) {
String contentType = ressponse.getEntity().getContentType().getValue();
if (contentType.contains("charset=")) {
return contentType.substring(contentType.indexOf("charset=") + 8);
}
}
return null;
}
/**
* 创建 SSL连接
* @return
* @throws GeneralSecurityException
*/
private static CloseableHttpClient createSSLInsecureClient() throws GeneralSecurityException {
try {
SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
public boolean isTrusted(X509Certificate[] chain,String authType) throws CertificateException {
return true;
}
}).build();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, new X509HostnameVerifier() {
@Override
public boolean verify(String arg0, SSLSession arg1) {
return true;
}
@Override
public void verify(String host, SSLSocket ssl)
throws IOException {
}
@Override
public void verify(String host, X509Certificate cert)
throws SSLException {
}
@Override
public void verify(String host, String[] cns,
String[] subjectAlts) throws SSLException {
}
});
return HttpClients.custom().setSSLSocketFactory(sslsf).build();
} catch (GeneralSecurityException e) {
throw e;
}
}
/**
* @param url http://taobao.com/test.action
* @param params 参数,编码之前的参数
* @return
* @throws IOException
* @throws UnsupportedEncodingException
* @throws ParseException
* @throws GeneralSecurityException
*/
public static String get(String url, Map<String, String> params,String charset, Integer connTimeout,Integer readTimeout) throws ParseException, UnsupportedEncodingException, IOException, GeneralSecurityException {
HttpClient client = null;
if(StringUtils.isBlank(url)){
return null;
}
if(params != null && !params.isEmpty()){
List<NameValuePair> pairs = new ArrayList<NameValuePair>(params.size());
for(Map.Entry<String,String> entry : params.entrySet()){
String value = entry.getValue();
if(value != null){
pairs.add(new BasicNameValuePair(entry.getKey(),value));
}
}
url += "?" + EntityUtils.toString(new UrlEncodedFormEntity(pairs, charset));
}
HttpGet httpget = new HttpGet(url);
CloseableHttpResponse response = null;
// 设置参数
Builder customReqConf = RequestConfig.custom();
if (connTimeout != null) {
customReqConf.setConnectTimeout(connTimeout);
}
if (readTimeout != null) {
customReqConf.setSocketTimeout(readTimeout);
}
httpget.setConfig(customReqConf.build());
if (url.startsWith("https")) {
logger.info("httpClientUtil|get|执行https的get请求|开始");
// 执行 Https 请求.
client = createSSLInsecureClient();
response = (CloseableHttpResponse) client.execute(httpget);
logger.info("httpClientUtil|get|执行https的get请求|结束");
} else {
logger.info("httpClientUtil|get|执行http的get请求|开始");
// 执行 Http 请求.
client = HttpClientUtil.client;
response = (CloseableHttpResponse) client.execute(httpget);
logger.info("httpClientUtil|get|执行http的get请求|结束");
}
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != 200) {
httpget.abort();
throw new RuntimeException("HttpClient,error status code :" + statusCode);
}
HttpEntity entity = response.getEntity();
String result = null;
if (entity != null) {
result = EntityUtils.toString(entity, "utf-8");
}
EntityUtils.consume(entity);
response.close();
return result;
}
}
服务端
HttpGetTestController代码如下：
package com.qhfax.controller.test;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpSession;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
/**
*
* HttpGet测试Controller
*
* @author zhanghaitao
*
*/
@Controller
@RequestMapping(value = "/httpGetTest")
public class HttpGetTestController {
/**
* 参数测试
*
* @param request 请求
* @param session 会话
* @return
*/
@RequestMapping(value = "/paramTest", method = RequestMethod.GET)
@ResponseBody
public String paramTest(HttpServletRequest request, HttpSession session) {
//获取签名
String sign = request.getParameter("sign");
System.out.println(sign);
//原签名
String oldSign = "abcde+fghij";
//验证服务端接收到的签名是否与客户端一致
boolean isEqual = oldSign.equals(sign);
return isEqual+"";
}
}