06-02 英文斷詞-共同單字.md

06-02 英文斷詞-共同單字

執行結果:

(1)main.py

from sklearn.feature_extraction.text import CountVectorizer

#--------------------------
# 讀入資料, text file
#--------------------------
f=open('doc01.txt', 'r', encoding='utf8')
doc01=f.read()

f=open('doc02.txt', 'r', encoding='utf8')
doc02=f.read()

print('原文')
print('<<文件1>>')
print(doc01.encode('raw_unicode_escape').decode('utf-8'))
print('<<文件2>>')
print(doc02.encode('raw_unicode_escape').decode('utf-8'))


#--------------------------
# 找出文章中的特徵文字
#--------------------------
vectorizer=CountVectorizer(min_df=1)
content=[]
content.append(doc01)
content.append(doc02)

stat=vectorizer.fit_transform(content)

print('特徵文字')
print(vectorizer.get_feature_names())

print('特徵文字統計')
print(stat.toarray().transpose())

print('兩者皆有的特徵文字統計')
fea=stat.toarray().transpose()
fea_names=vectorizer.get_feature_names()
k=0
for i, j in fea:
    k=k+1
    if i>0 and j>0:
        print(i, j)
        print(fea_names[k])        

(2)doc01.txt

OAKLAND, Calif. (AP) Stephen Curry dribbled every which way and beat LeBron James to the rim in a move reminiscent of his recent MVP magic, and the Golden State Warriors grabbed a 2-0 lead in the NBA Finals by downing the Cleveland Cavaliers 132-113 Sunday night in coach Steve Kerr’s return to the sideline after a six-week absence.

Their leader and reigning NBA Coach of the Year back on the bench at last, Curry and Kevin Durant tag-teamed the Warriors within two victories of another championship. Curry recorded his first career postseason triple-double with 32 points, 11 assists and 10 rebounds, while Durant contributed 33 points, 13 rebounds and six assists while moving two wins from his first ring.

(3)doc02.txt

James was equally spectacular with his own triple-double of 29 points, 14 assists and 11 rebounds, but yet again didn’t get nearly enough help to keep up with Golden State’s depth despite Kevin Love’s 27 points and 19 from Kyrie Irving.

Klay Thompson emerged from a postseason-long shooting funk to score 22 points with four 3-pointers while playing more stellar defense and pulling down seven rebounds for the Warriors, who shot 18 for 43 from deep.

”We turned the ball over too much in the first half. But the second half, I think we settled in, tried to play simple and defensively just tried to contest their shots,” Durant said.

The best-of-seven series now shifts to Cleveland for two games, with Game 3 Wednesday.