<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Distributed Representation on Subculture Science Research Group&#39;s Blog</title>
    <link>https://www.subcul-science.com/en/tags/distributed-representation/</link>
    <description>Recent content in Distributed Representation on Subculture Science Research Group&#39;s Blog</description>
    <generator>Hugo</generator>
    <language>en-US</language>
    <lastBuildDate>Wed, 19 May 2021 02:10:01 +0900</lastBuildDate>
    <atom:link href="https://www.subcul-science.com/en/tags/distributed-representation/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Why is fasttext so fast?</title>
      <link>https://www.subcul-science.com/en/post/20210519fasttext/</link>
      <pubDate>Wed, 19 May 2021 02:10:01 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20210519fasttext/</guid>
      <description>&lt;h2 id=&#34;features-of-fasttext&#34;&gt;Features of fasttext&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Improved objective function&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Consideration of negative samples&lt;/li&gt;&#xA;&lt;li&gt;This should not affect training time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Change in optimization method&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use of stochastic optimization&lt;/li&gt;&#xA;&lt;li&gt;If it affects the learning time, it should be this one&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Implementation in C language&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is the most effective, isn&amp;rsquo;t it?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If we implement it in ## pytorch, it won&amp;rsquo;t be much different from word2vec.&#xA;It would depend on the amount of data to be trained.&lt;/p&gt;</description>
    </item>
    <item>
      <title>On the use of distributed representations bagging for class classification and generalization performance</title>
      <link>https://www.subcul-science.com/en/post/20210204classifier/</link>
      <pubDate>Thu, 04 Feb 2021 13:24:19 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20210204classifier/</guid>
      <description>&lt;p&gt;After the distributed representation has been obtained, the&#xA;After the distributed representation is obtained, machine learning can be used to classify it.&lt;/p&gt;&#xA;&lt;p&gt;Models that can be used include&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decision Tree&lt;/li&gt;&#xA;&lt;li&gt;SVM Support Vector Machine&lt;/li&gt;&#xA;&lt;li&gt;NN Neural Networks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;and others.&lt;/p&gt;&#xA;&lt;p&gt;SVM is included in NN in a broad sense.&lt;/p&gt;&#xA;&lt;p&gt;In this section, we will use the decision tree method.&lt;/p&gt;&#xA;&lt;h2 id=&#34;bagging&#34;&gt;Bagging&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Image of majority voting with multiple decision trees&lt;/li&gt;&#xA;&lt;li&gt;Simple theory&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decision trees are highly explainable and are a classic machine learning model.&lt;/li&gt;&#xA;&lt;li&gt;Computational load is light compared to deep learning&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Depends on the size of the model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Not much explainability&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do we want to analyze each of the multiple decision trees?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;``py&#xA;from sklearn.ensemble import BaggingClassifier&#xA;from sklearn.tree import DecisionTreeClassifier&lt;/p&gt;</description>
    </item>
    <item>
      <title>How to train a Japanese model with Sentence transformer to get a distributed representation of a sentence</title>
      <link>https://www.subcul-science.com/en/post/20210203sbert/</link>
      <pubDate>Wed, 03 Feb 2021 15:58:41 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20210203sbert/</guid>
      <description>&lt;p&gt;.&#xA;BERT is a model that can be powerfully applied to natural language processing tasks.&lt;/p&gt;&#xA;&lt;p&gt;However, it does not do a good job of capturing sentence-wise features.&lt;/p&gt;&#xA;&lt;p&gt;Some claim that sentence features appear in [ CLS\ ], but&#xA;This paper](&lt;a href=&#34;https://arxiv.org/abs/1908.10084&#34;&gt;https://arxiv.org/abs/1908.10084&lt;/a&gt;) claims that it does not contain that much useful information for the task.&lt;/p&gt;&#xA;&lt;p&gt;Sentence BERT is a model that extends BERT to be able to obtain features per sentence.&lt;/p&gt;&#xA;&lt;p&gt;The following are the steps to create Sentence BERT in Japanese.&lt;/p&gt;</description>
    </item>
    <item>
      <title>Using BART (sentence summary model) with hugging face</title>
      <link>https://www.subcul-science.com/en/post/20210119huggingface/</link>
      <pubDate>Tue, 19 Jan 2021 03:03:11 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20210119huggingface/</guid>
      <description>&lt;ul&gt;&#xA;&lt;li&gt;BART is a model for document summarization&lt;/li&gt;&#xA;&lt;li&gt;Derived from the same transformer as BERT&lt;/li&gt;&#xA;&lt;li&gt;Unlike BERT, it has an encoder-decoder structure&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is because it is intended for sentence generation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This page shows the steps to run a tutorial on BART.&lt;/p&gt;&#xA;&lt;h2 id=&#34;procedure&#34;&gt;Procedure&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;install transformers&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Run ``sh&#xA;pip install transformers&lt;/p&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-fallback&#34; data-lang=&#34;fallback&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Run summary&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;2. Run the summary&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;from transformers import BartTokenizer, BartForConditionalGeneration, BartConfig&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;model = BartForConditionalGeneration.from_pretrained(&amp;#39;facebook/bart-large&amp;#39;)&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;tokenizer = BartTokenizer.from_pretrained(&amp;#39;facebook/bart-large&amp;#39;)&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;ARTICLE_TO_SUMMARIZE = &amp;#34;My friends are cool but they eat too many carbs.&amp;#34;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors=&amp;#39;pt&amp;#39;)&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;# Generate Summary&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;summary_ids = model.generate(inputs[&amp;#39;input_ids&amp;#39;], num_beams=4, max_length=5, early_stopping=True)&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;```&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;On 2021/01/18, the output was MyMy friends.&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Interesting.&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;## Where I got stuck.&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;Error when the version of pytorch is different from the one specified in transformers.&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;pip install -U torch&lt;/p&gt;</description>
    </item>
    <item>
      <title>Procedure for obtaining a distributed representation of a Japanese sentence using a trained Universal Sentence Encoder</title>
      <link>https://www.subcul-science.com/en/post/universal-sentence-encoder/</link>
      <pubDate>Mon, 22 Jun 2020 18:29:00 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/universal-sentence-encoder/</guid>
      <description>&lt;p&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A vector of documents can be obtained using Universal Sentence Encoder.&lt;/p&gt;&#xA;&lt;h2 id=&#34;features&#34;&gt;Features&lt;/h2&gt;&#xA;&lt;p&gt;Supports multiple languages.&lt;/p&gt;&#xA;&lt;p&gt;Japanese is supported.&lt;/p&gt;&#xA;&lt;p&gt;Can handle Japanese sentences as vectors.&lt;/p&gt;&#xA;&lt;h2 id=&#34;usage&#34;&gt;Usage&lt;/h2&gt;&#xA;&lt;p&gt;Clustering, similarity calculation, feature extraction.&lt;/p&gt;&#xA;&lt;h2 id=&#34;usage-1&#34;&gt;Usage&lt;/h2&gt;&#xA;&lt;p&gt;Execute the following command as preparation.&lt;/p&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-sh&#34; data-lang=&#34;sh&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;pip install tensorflow tensorflow_hub tensorflow_text numpy   &#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Trained models are available.&lt;/p&gt;&#xA;&lt;p&gt;See the python description below for details on how to use it.&lt;/p&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; class=&#34;chroma&#34;&gt;&lt;code class=&#34;language-py&#34; data-lang=&#34;py&#34;&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;tensorflow_hub&lt;/span&gt; &lt;span class=&#34;k&#34;&gt;as&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;hub&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;tensorflow_text&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;numpy&lt;/span&gt; &lt;span class=&#34;k&#34;&gt;as&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;np&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;# for avoiding error  &lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;ssl&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;ssl&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;_create_default_https_context&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;ssl&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;_create_unverified_context&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;k&#34;&gt;def&lt;/span&gt; &lt;span class=&#34;nf&#34;&gt;cos_sim&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;v2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;):&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;   &lt;span class=&#34;k&#34;&gt;return&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;np&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;dot&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;v2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;/&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;np&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;linalg&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;norm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;v1&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;*&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;np&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;linalg&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;norm&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;v2&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;))&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;embed&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;hub&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;load&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;https://tfhub.dev/google/universal-sentence-encoder-multilingual/3&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;texts&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;s2&#34;&gt;&amp;#34;I saw a comedy show yesterday.&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;There was a comedy show on TV last night.&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;I went to the park yesterday.&amp;#34;&lt;/span&gt; &lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;I saw a comedy show last night.&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;s2&#34;&gt;&amp;#34;Yesterday, I went to the park.&amp;#34;&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;]&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;vectors&lt;/span&gt; &lt;span class=&#34;o&#34;&gt;=&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;embed&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;texts&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;  &#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;```&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;n&#34;&gt;See&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;the&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;following&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;link&lt;/span&gt; &lt;span class=&#34;k&#34;&gt;for&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;more&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;details&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;Try&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Universal&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Sentence&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Encoder&lt;/span&gt; &lt;span class=&#34;ow&#34;&gt;in&lt;/span&gt; &lt;span class=&#34;n&#34;&gt;Japanese&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;](&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;https&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;//&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;qiita&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;com&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;kenta1984&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;items&lt;/span&gt;&lt;span class=&#34;o&#34;&gt;/&lt;/span&gt;&lt;span class=&#34;mi&#34;&gt;9613&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;da23766a2578a27a&lt;/span&gt;&lt;span class=&#34;p&#34;&gt;)&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;c1&#34;&gt;### Postscript&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;err&#34;&gt;```&lt;/span&gt;&lt;span class=&#34;n&#34;&gt;py&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;line&#34;&gt;&lt;span class=&#34;cl&#34;&gt;&lt;span class=&#34;kn&#34;&gt;import&lt;/span&gt; &lt;span class=&#34;nn&#34;&gt;tensorflow_text&lt;/span&gt;&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Without this line, you will get an error like ``Sentencepiece not found! error.&#xA;This line is not explicitly used in the sample source, but is required for the actual execution.&#xA;This line is not explicitly used in the sample source, but is required in the actual runtime.&lt;/p&gt;</description>
    </item>
    <item>
      <title>A note on how to use BERT learned from Japanese Wikipedia, now available</title>
      <link>https://www.subcul-science.com/en/post/wikipediabert/</link>
      <pubDate>Wed, 17 Jun 2020 07:34:00 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/wikipediabert/</guid>
      <description>&lt;p&gt;huggingface has released a Japanese model for BERT.&lt;/p&gt;&#xA;&lt;p&gt;The Japanese model is included in transformers.&lt;/p&gt;&#xA;&lt;p&gt;However, I stumbled over a few things before I could get it to actually work in a Mac environment, so I&amp;rsquo;ll leave a note.&lt;/p&gt;&#xA;&lt;h2 id=&#34;preliminaries-installing-mecab&#34;&gt;Preliminaries: Installing mecab&lt;/h2&gt;&#xA;&lt;p&gt;The morphological analysis engine, mecab, is required to use BERT&amp;rsquo;s Japanese model.&lt;/p&gt;&#xA;&lt;p&gt;The tokenizer will probably ask for mecab.&lt;/p&gt;&#xA;&lt;p&gt;This time, we will use homebrew to install Mecab and ipadic.&lt;/p&gt;</description>
    </item>
    <item>
      <title>I even did a document classification problem with Fasttext</title>
      <link>https://www.subcul-science.com/en/post/20200613fasttext/</link>
      <pubDate>Sat, 13 Jun 2020 07:29:00 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20200613fasttext/</guid>
      <description>&lt;h2 id=&#34;summary-of-what-ive-done-with-fasttext-to-the-document-classification-problem&#34;&gt;Summary of what I&amp;rsquo;ve done with Fasttext to the document classification problem.&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Facebook research has published a document classification library using Fasttext.&lt;/li&gt;&#xA;&lt;li&gt;Fasttext is easy to install in a python environment.&lt;/li&gt;&#xA;&lt;li&gt;Run time is fast.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2 id=&#34;preliminaries&#34;&gt;Preliminaries&lt;/h2&gt;&#xA;&lt;p&gt;I decided to tackle the task of document classification, and initially thought.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit. However, it was not very accurate.&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
