get_all_tweets with any mentioned users data field

[renanpwagner]

Hi, I was wondering if it is possible to add a new data_column request on the get_all_tweets containing the "Any mentioned users’ object" (entities.mentions.username)

I assume this would make importing the binded csv into Gephi more straightforward in order to do network analyses (e.g., the edge lists would correspond to that call/request).

Correct if I am wrong (very new to this....)

[Dominikelisa]

Hi,
I agree with this! Meaning it would be great if this could be part of the tidy format when binding the tweets. So far, user mentions are accessible through the untidy or the raw format in entities$mentions.
Does anyone have a nice work around (code) to share how to unnest entities$mentions to create seperate data columns to later use this for network analysis?

[Dominikelisa]

This is probably not the most elegant way, but it does its job. Mentions here include retweets and replies as one automatically mentions another user when rt/replying.

# Load tweets 
tweets_bind_df <- bind_tweets(data_path = "tweetsdf/") %>%  #for data stored in a data_path
  as_tibble()
tweets_bind_df_tidy <- bind_tweets(data_path = "tweetsdf/", output_format = "tidy")

# Unnest dataframe column entities
unnested_df <- unnest(tweets_bind_df, cols = entities, keep_empty = TRUE)
entities_df <- tweets_bind_df$entities %>%
  as_tibble()
# Extract mentions column as list
mentions_list <- entities_df$mentions

### If element in list is an empty list replace it with an empty dataframe
# Empty dataframe
empty_df <-
  data.frame(start = NA,
             end = NA,
             username = NA,
             id = NA)
# Loop
for (i in seq_along(mentions_list)) { # loop over elements in list
  if (!is.data.frame(mentions_list[[i]])) { # check if not a dataframe
    mentions_list[[i]] <- empty_df # replace with empty dataframe
  }
}

# Replace column with empty lists with column with empty dataframes
unnested_df$mentions <- mentions_list

# Unnest mentions column to longer (i.e. one or more rows per tweet_id)
tweets_bind_df_longer <- unnest_longer(unnested_df, col = mentions)


# Rename mentions columns and unnest mentions df
names(tweets_bind_df_longer$mentions)[names(tweets_bind_df_longer$mentions) == 'id'] <- 'mentions_id'
names(tweets_bind_df_longer$mentions)[names(tweets_bind_df_longer$mentions) == 'username'] <- 'mentions_username'

tweets_bind_df_mentions <- tweets_bind_df_longer %>% unnest(mentions)

# Join by tweet_id
names(tweets_bind_df_mentions)[names(tweets_bind_df_mentions) == 'id'] <- 'tweet_id'
tweets_bind_df_mentions_narrow <- 
  tweets_bind_df_mentions[,c("tweet_id", "mentions_username", 
                                                  "mentions_id")]

tweets_bind_mentions_tidy_df <- 
  left_join(tweets_bind_df_tidy, tweets_bind_df_mentions_narrow, by='tweet_id')