本文共 4748 字,大约阅读时间需要 15 分钟。
pom.xml引入中文分词器IKAnalyzer
com.janeluo ikanalyzer 2012_u6 org.apache.lucene lucene-core org.apache.lucene lucene-queryparser org.apache.lucene lucene-analyzers-common
IKAnalyzer4Lucene7:
package com.dalomao.framework.lucene.analizer.ik;import org.apache.lucene.analysis.Analyzer;public class IKAnalyzer4Lucene7 extends Analyzer { private boolean useSmart = false; public IKAnalyzer4Lucene7() { this(false); } public IKAnalyzer4Lucene7(boolean useSmart) { super(); this.useSmart = useSmart; } public boolean isUseSmart() { return useSmart; } public void setUseSmart(boolean useSmart) { this.useSmart = useSmart; } @Override protected TokenStreamComponents createComponents(String fieldName) { IKTokenizer4Lucene7 tk = new IKTokenizer4Lucene7(this.useSmart); return new TokenStreamComponents(tk); }}
IKTokenizer4Lucene7:
package com.dalomao.framework.lucene.analizer.ik;import org.apache.lucene.analysis.Tokenizer;import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;import org.apache.lucene.analysis.tokenattributes.OffsetAttribute;import org.apache.lucene.analysis.tokenattributes.TypeAttribute;import org.wltea.analyzer.core.IKSegmenter;import org.wltea.analyzer.core.Lexeme;import java.io.IOException;public class IKTokenizer4Lucene7 extends Tokenizer { // IK分词器实现 private IKSegmenter _IKImplement; // 词元文本属性 private final CharTermAttribute termAtt; // 词元位移属性 private final OffsetAttribute offsetAtt; // 词元分类属性(该属性分类参考org.wltea.analyzer.core.Lexeme中的分类常量) private final TypeAttribute typeAtt; // 记录最后一个词元的结束位置 private int endPosition; /** * @param * @param useSmart */ public IKTokenizer4Lucene7(boolean useSmart) { super(); offsetAtt = addAttribute(OffsetAttribute.class); termAtt = addAttribute(CharTermAttribute.class); typeAtt = addAttribute(TypeAttribute.class); _IKImplement = new IKSegmenter(input, useSmart); } /* * (non-Javadoc) * * @see org.apache.lucene.analysis.TokenStream#incrementToken() */ @Override public boolean incrementToken() throws IOException { // 清除所有的词元属性 clearAttributes(); Lexeme nextLexeme = _IKImplement.next(); if (nextLexeme != null) { // 将Lexeme转成Attributes // 设置词元文本 termAtt.append(nextLexeme.getLexemeText()); // 设置词元长度 termAtt.setLength(nextLexeme.getLength()); // 设置词元位移 offsetAtt.setOffset(nextLexeme.getBeginPosition(), nextLexeme.getEndPosition()); // 记录分词的最后位置 endPosition = nextLexeme.getEndPosition(); // 记录词元分类 typeAtt.setType(nextLexeme.getLexemeTypeString()); // 返会true告知还有下个词元 return true; } // 返会false告知词元输出完毕 return false; } /* * (non-Javadoc) * * @see org.apache.lucene.analysis.Tokenizer#reset(java.io.Reader) */ @Override public void reset() throws IOException { super.reset(); _IKImplement.reset(input); } @Override public final void end() { // set final offset int finalOffset = correctOffset(this.endPosition); offsetAtt.setOffset(finalOffset, finalOffset); }}
IKTokenizer4Lucene7Factory:
package com.dalomao.framework.lucene.analizer.ik;import java.util.Map;import org.apache.lucene.analysis.Tokenizer;import org.apache.lucene.analysis.util.TokenizerFactory;import org.apache.lucene.util.AttributeFactory;public class IKTokenizer4Lucene7Factory extends TokenizerFactory{ private boolean useSmart = false; public IKTokenizer4Lucene7Factory(Mapargs) { super(args); String useSmartParm = (String)args.get("useSmart"); if ("true".equalsIgnoreCase(useSmartParm)) this.useSmart = true; } public Tokenizer create(AttributeFactory factory) { return new IKTokenizer4Lucene7(this.useSmart); }}
扩展词:lucene_ext.dic
厉害了我的国
停用词:lucene_ext_stopword.dic
,?、我的你
IK配置文件:IKAnalyzer.cfg.xml
IK Analyzer 扩展配置 lucene_ext.dic lucene_ext_stopword.dic
将前面定义的三个类达成jar包,命名为IKAnalyzer-lucene7.5.jar。
备注:具体如何打包请参看将这个IKAnalyzer-lucene7.5.jar和 IKAnalyzer的jar(ikanalyzer-2012_u6.jar)拷贝到solr的web应用的lib目录下(solr-7.5.0/server/solr-webapp/webapp/WEB-INF/lib)
将之前定义的三个配置文件拷贝到solr安装目录的应用classes目录下( solr-7.3.0/server/solr-webapp/webapp/WEB-INF/classes),如果没有classes目录则需要新建
在内核主目录下conf/managed-schema.xml文件中定义一个新的FieldType,如下:
以上全部步骤做完以后,重启Solr服务器实例,然后登陆web测试
测试成功!转载地址:http://sgsxi.baihongyu.com/