Twitter4J Extended Mode – 280 Character Tweets

Looking to use Twitter4J to download the full 280 characters of extended tweets? I can easily be done. You need to have a relatively recent version to enable extended mode, I have tested this with version 4.0.6 and it works well.

Its pretty straightforward and you enabled it at the configuration builder phase.

twitter4j.conf.ConfigurationBuilder()
      .setOAuthConsumerKey(""))
      .setOAuthConsumerSecret("")
      .setOAuthAccessToken("")
      .setOAuthAccessTokenSecret("")
      .setTweetModeExtended(true)

Thats is, the setTweetModeExtended(true) is all you need.

Graph Visualisation with Neo4J and Alchemy

I have been using the Alchemy.js library to do some graph visualisation for netorks. Its pretty good, my default template is below as I had some trouble with those I found on around the internet.

It makes a simple graph with node labels visible, and takes the data as a .json file. I produce the data file from an R script that extracts data out of the Neo4J database. The basic code for that is below. One thing to remember with this is that the data file with the network in is easily accessible to anyone that might want to download it. So if it constitutes a lot of work and you don’t want anyone to get hold of it (e.g. if it is research you intend to publish at a later point), this isn’t the method for you.

<html>
<head>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/alchemyjs/0.4.2/alchemy.min.css" />
</head>
<body>
    <h1>Network Visualisation</h1>
  
    <div class="alchemy" id="alchemy"></div>

    <script src="http://cdn.graphalchemist.com/alchemy.min.js"></script>

    <script src="https://d3js.org/d3.v3.min.js" charset="utf-8"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/alchemyjs/0.4.2/alchemy.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/alchemyjs/0.4.2/scripts/vendor.js"></script>
  
    <script type="text/javascript">
         var config = {
              dataSource: 'data.json',
              forceLocked: false,
              graphHeight: function(){ return 1000; },
              graphWidth: function(){ return 1000; },      
              linkDistance: function(){ return 40; },
              nodeCaptionsOnByDefault: true,
              "nodeCaption": 'name', 
              "nodeMouseOver": 'name'
            
          };

          alchemy = new Alchemy(config)
     </script>
</body>
</html> 

Here is the R code that generates json data file from the Neo4J database, its really simple. Connect to the local running Neo4J database, run a query to get the nodes, run a query to get the relationships (edges), and then I did some quick clustering analysis in iGraph. You need to do a bit of work to get the communities into the dataframe, if you cluster. You then need to do a little work to get the json file. Its pretty good an not a lot of code to get a nice visualisation from your Neo4J database.

library(RNeo4j)
library(igraph)
neo4j = startGraph("http://localhost:7474/db/data/")

nodes_query = "MATCH (a) RETURN DISTINCT ID(a) AS id, a.name AS name, labels(a) AS type"
nodes = cypher(neo4j, nodes_query)

rels_query = "MATCH (a1)--(a2) RETURN ID(a1) AS source, ID(a2) AS target"
rels = cypher(neo4j, rels_query)

ig = graph.data.frame(rels,directed=TRUE,nodes)
communities = edge.betweenness.community(ig)

memb = data.frame(name = communities$names, cluster = communities$membership)
nodes = merge(nodes, memb)

nodes_json = paste0("\"nodes\":", jsonlite::toJSON(nodes))
edges_json = paste0("\"edges\":", jsonlite::toJSON(rels))
all_json = paste0("{", nodes_json, ",", edges_json, "}")

sink(file = 'data.json')
cat(all_json)
sink()

Pushed Code to GitHub – Non Exact String Matching

I published my first project to GitHub! Up till now I have been a user of SVN but I have long wanted to publish some useful bits of code. I have done my first one! Its a non exact string matching program. Its pretty rough round the edges and in need of some updating but now all updates will be to the benefit of anyone that has an interest in them.

ObscurusCode is now an organization on GitHub too!